Note: This is test shard 7 of 8.
[==========] Running 35 tests from 9 test suites.
[----------] Global test environment set-up.
[----------] 20 tests from ClientTest
[ RUN ] ClientTest.ConnectToClusterNoMasterAddressSpecified
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251212 08:52:37.245829 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:36649
I20251212 08:52:37.248436 832 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251212 08:52:37.249956 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:37.263547 838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:37.263597 840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:37.263782 832 server_base.cc:1047] running on GCE node
W20251212 08:52:37.263629 837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:37.265141 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:37.265295 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:37.265374 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529557265358 us; error 0 us; skew 500 ppm
I20251212 08:52:37.265700 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:37.268733 832 webserver.cc:492] Webserver started at http://127.0.208.62:46721/ using document root <none> and password file <none>
I20251212 08:52:37.269474 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:37.269570 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:37.269937 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:37.273401 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "3dc795ad7a694383900eb24b825038ee"
format_stamp: "Formatted at 2025-12-12 08:52:37 on dist-test-slave-qdpb"
I20251212 08:52:37.280135 832 fs_manager.cc:696] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20251212 08:52:37.283874 845 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:37.285673 832 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:37.285842 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "3dc795ad7a694383900eb24b825038ee"
format_stamp: "Formatted at 2025-12-12 08:52:37 on dist-test-slave-qdpb"
I20251212 08:52:37.286015 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:37.308516 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:37.309495 832 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251212 08:52:37.309772 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:37.339882 906 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:36649 every 8 connection(s)
I20251212 08:52:37.339893 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:36649
I20251212 08:52:37.345643 907 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:37.359009 907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee: Bootstrap starting.
I20251212 08:52:37.363063 907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:37.364327 907 log.cc:826] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee: Log is configured to *not* fsync() on all Append() calls
I20251212 08:52:37.367425 907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee: No bootstrap required, opened a new log
I20251212 08:52:37.374778 907 raft_consensus.cc:359] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc795ad7a694383900eb24b825038ee" member_type: VOTER }
I20251212 08:52:37.375743 907 raft_consensus.cc:385] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:37.375901 907 raft_consensus.cc:740] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3dc795ad7a694383900eb24b825038ee, State: Initialized, Role: FOLLOWER
I20251212 08:52:37.376787 907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc795ad7a694383900eb24b825038ee" member_type: VOTER }
I20251212 08:52:37.377046 907 raft_consensus.cc:399] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:37.377159 907 raft_consensus.cc:493] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:37.377336 907 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:37.379642 907 raft_consensus.cc:515] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc795ad7a694383900eb24b825038ee" member_type: VOTER }
I20251212 08:52:37.380226 907 leader_election.cc:304] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3dc795ad7a694383900eb24b825038ee; no voters:
I20251212 08:52:37.380880 907 leader_election.cc:290] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:37.381052 910 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:37.381841 910 raft_consensus.cc:697] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 1 LEADER]: Becoming Leader. State: Replica: 3dc795ad7a694383900eb24b825038ee, State: Running, Role: LEADER
I20251212 08:52:37.382437 910 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc795ad7a694383900eb24b825038ee" member_type: VOTER }
I20251212 08:52:37.383227 907 sys_catalog.cc:565] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:37.385838 911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3dc795ad7a694383900eb24b825038ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc795ad7a694383900eb24b825038ee" member_type: VOTER } }
I20251212 08:52:37.386154 911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:37.387264 912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3dc795ad7a694383900eb24b825038ee. Latest consensus state: current_term: 1 leader_uuid: "3dc795ad7a694383900eb24b825038ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3dc795ad7a694383900eb24b825038ee" member_type: VOTER } }
I20251212 08:52:37.387658 912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:37.389760 919 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:37.396034 919 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:37.396740 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:37.406832 919 catalog_manager.cc:1357] Generated new cluster ID: 70547e0e579b493b9004cf57b91f3892
I20251212 08:52:37.406982 919 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:37.436792 919 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:37.438596 919 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:37.458389 919 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee: Generated new TSK 0
I20251212 08:52:37.459589 919 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:37.462527 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:37.467758 930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:37.467734 929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:37.469115 932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:37.469672 832 server_base.cc:1047] running on GCE node
I20251212 08:52:37.469988 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:37.470086 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:37.470149 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529557470118 us; error 0 us; skew 500 ppm
I20251212 08:52:37.470389 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:37.472008 832 webserver.cc:492] Webserver started at http://127.0.208.1:42773/ using document root <none> and password file <none>
I20251212 08:52:37.472365 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:37.472476 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:37.472669 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:37.474637 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "d338ee517e324b0db9a47904a878cbef"
format_stamp: "Formatted at 2025-12-12 08:52:37 on dist-test-slave-qdpb"
I20251212 08:52:37.478722 832 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20251212 08:52:37.481029 937 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:37.481483 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.001s
I20251212 08:52:37.481612 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "d338ee517e324b0db9a47904a878cbef"
format_stamp: "Formatted at 2025-12-12 08:52:37 on dist-test-slave-qdpb"
I20251212 08:52:37.481741 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.ConnectToClusterNoMasterAddressSpecified.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:37.498137 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:37.498836 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:37.501719 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:37.501845 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:37.502008 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:37.502712 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:37.585098 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:44283
I20251212 08:52:37.585165 1007 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:44283 every 8 connection(s)
I20251212 08:52:37.588052 1008 heartbeater.cc:344] Connected to a master server at 127.0.208.62:36649
I20251212 08:52:37.588486 1008 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:37.589411 1008 heartbeater.cc:507] Master 127.0.208.62:36649 requested a full tablet report, sending...
I20251212 08:52:37.591758 862 ts_manager.cc:194] Registered new tserver with Master: d338ee517e324b0db9a47904a878cbef (127.0.208.1:44283)
I20251212 08:52:37.591921 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.004090296s
I20251212 08:52:37.594933 862 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38790
I20251212 08:52:37.608201 1008 heartbeater.cc:499] Master 127.0.208.62:36649 was elected leader, sending a full tablet report...
I20251212 08:52:37.611145 862 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38808:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:37.641347 967 tablet_service.cc:1505] Processing CreateTablet for tablet b48e3d6796024165acb74d34cafaf82f (DEFAULT_TABLE table=client-testtb [id=f4911b9e29644930bee5a9288e89f6a6]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:37.641396 970 tablet_service.cc:1505] Processing CreateTablet for tablet a37cd3638f8d4061ae410930f371d0e6 (DEFAULT_TABLE table=client-testtb [id=f4911b9e29644930bee5a9288e89f6a6]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:37.643059 967 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b48e3d6796024165acb74d34cafaf82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:37.643337 970 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a37cd3638f8d4061ae410930f371d0e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:37.651618 1018 tablet_bootstrap.cc:492] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef: Bootstrap starting.
I20251212 08:52:37.654099 1018 tablet_bootstrap.cc:654] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:37.656347 1018 tablet_bootstrap.cc:492] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef: No bootstrap required, opened a new log
I20251212 08:52:37.656539 1018 ts_tablet_manager.cc:1403] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s
I20251212 08:52:37.657313 1018 raft_consensus.cc:359] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } }
I20251212 08:52:37.657519 1018 raft_consensus.cc:385] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:37.657691 1018 raft_consensus.cc:740] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d338ee517e324b0db9a47904a878cbef, State: Initialized, Role: FOLLOWER
I20251212 08:52:37.657939 1018 consensus_queue.cc:260] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } }
I20251212 08:52:37.658105 1018 raft_consensus.cc:399] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:37.658231 1018 raft_consensus.cc:493] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:37.658381 1018 raft_consensus.cc:3060] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:37.661226 1018 raft_consensus.cc:515] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } }
I20251212 08:52:37.661471 1018 leader_election.cc:304] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d338ee517e324b0db9a47904a878cbef; no voters:
I20251212 08:52:37.662001 1018 leader_election.cc:290] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:37.662788 1020 raft_consensus.cc:2804] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:37.663620 1018 ts_tablet_manager.cc:1434] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20251212 08:52:37.663972 1020 raft_consensus.cc:697] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 1 LEADER]: Becoming Leader. State: Replica: d338ee517e324b0db9a47904a878cbef, State: Running, Role: LEADER
I20251212 08:52:37.664443 1018 tablet_bootstrap.cc:492] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef: Bootstrap starting.
I20251212 08:52:37.664366 1020 consensus_queue.cc:237] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } }
I20251212 08:52:37.666572 1018 tablet_bootstrap.cc:654] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:37.669317 861 catalog_manager.cc:5654] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef reported cstate change: term changed from 0 to 1, leader changed from <none> to d338ee517e324b0db9a47904a878cbef (127.0.208.1). New cstate: current_term: 1 leader_uuid: "d338ee517e324b0db9a47904a878cbef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:37.681551 1018 tablet_bootstrap.cc:492] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef: No bootstrap required, opened a new log
I20251212 08:52:37.681757 1018 ts_tablet_manager.cc:1403] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.007s
I20251212 08:52:37.682608 1018 raft_consensus.cc:359] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } }
I20251212 08:52:37.682838 1018 raft_consensus.cc:385] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:37.682940 1018 raft_consensus.cc:740] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d338ee517e324b0db9a47904a878cbef, State: Initialized, Role: FOLLOWER
I20251212 08:52:37.683235 1018 consensus_queue.cc:260] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } }
I20251212 08:52:37.683444 1018 raft_consensus.cc:399] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:37.683671 1018 raft_consensus.cc:493] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:37.683826 1018 raft_consensus.cc:3060] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:37.686192 1018 raft_consensus.cc:515] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } }
I20251212 08:52:37.686451 1018 leader_election.cc:304] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d338ee517e324b0db9a47904a878cbef; no voters:
I20251212 08:52:37.686702 1018 leader_election.cc:290] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:37.686776 1021 raft_consensus.cc:2804] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:37.687227 1018 ts_tablet_manager.cc:1434] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20251212 08:52:37.687351 1021 raft_consensus.cc:697] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 1 LEADER]: Becoming Leader. State: Replica: d338ee517e324b0db9a47904a878cbef, State: Running, Role: LEADER
I20251212 08:52:37.687678 1021 consensus_queue.cc:237] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } }
I20251212 08:52:37.689980 861 catalog_manager.cc:5654] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef reported cstate change: term changed from 0 to 1, leader changed from <none> to d338ee517e324b0db9a47904a878cbef (127.0.208.1). New cstate: current_term: 1 leader_uuid: "d338ee517e324b0db9a47904a878cbef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d338ee517e324b0db9a47904a878cbef" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44283 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:37.706612 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:52:37.721572 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:52:37.722371 832 tablet_replica.cc:333] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef: stopping tablet replica
I20251212 08:52:37.722852 832 raft_consensus.cc:2243] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:37.723158 832 raft_consensus.cc:2272] T a37cd3638f8d4061ae410930f371d0e6 P d338ee517e324b0db9a47904a878cbef [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:37.725188 832 tablet_replica.cc:333] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef: stopping tablet replica
I20251212 08:52:37.725489 832 raft_consensus.cc:2243] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:37.725745 832 raft_consensus.cc:2272] T b48e3d6796024165acb74d34cafaf82f P d338ee517e324b0db9a47904a878cbef [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:37.742580 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:52:37.751540 832 master.cc:561] Master@127.0.208.62:36649 shutting down...
I20251212 08:52:37.770329 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:37.770686 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:37.770989 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 3dc795ad7a694383900eb24b825038ee: stopping tablet replica
I20251212 08:52:37.785990 832 master.cc:583] Master@127.0.208.62:36649 shutdown complete.
[ OK ] ClientTest.ConnectToClusterNoMasterAddressSpecified (556 ms)
[ RUN ] ClientTest.TestScanAtFutureTimestamp
I20251212 08:52:37.797708 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:44445
I20251212 08:52:37.798292 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:37.801035 1031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:37.801229 1032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:37.802778 1034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:37.803459 832 server_base.cc:1047] running on GCE node
I20251212 08:52:37.803740 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:37.803833 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:37.803902 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529557803878 us; error 0 us; skew 500 ppm
I20251212 08:52:37.804095 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:37.805521 832 webserver.cc:492] Webserver started at http://127.0.208.62:42331/ using document root <none> and password file <none>
I20251212 08:52:37.805778 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:37.805867 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:37.806015 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:37.806682 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "d5aebce2852341ff8abb004c161dcd5e"
format_stamp: "Formatted at 2025-12-12 08:52:37 on dist-test-slave-qdpb"
I20251212 08:52:37.809391 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20251212 08:52:37.811144 1039 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:37.811524 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:37.811663 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "d5aebce2852341ff8abb004c161dcd5e"
format_stamp: "Formatted at 2025-12-12 08:52:37 on dist-test-slave-qdpb"
I20251212 08:52:37.811782 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:37.827107 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:37.827807 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:37.850884 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:44445
I20251212 08:52:37.850900 1100 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:44445 every 8 connection(s)
I20251212 08:52:37.852599 1101 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:37.857219 1101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e: Bootstrap starting.
I20251212 08:52:37.859402 1101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:37.861231 1101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e: No bootstrap required, opened a new log
I20251212 08:52:37.861953 1101 raft_consensus.cc:359] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aebce2852341ff8abb004c161dcd5e" member_type: VOTER }
I20251212 08:52:37.862102 1101 raft_consensus.cc:385] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:37.862197 1101 raft_consensus.cc:740] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5aebce2852341ff8abb004c161dcd5e, State: Initialized, Role: FOLLOWER
I20251212 08:52:37.862433 1101 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aebce2852341ff8abb004c161dcd5e" member_type: VOTER }
I20251212 08:52:37.862588 1101 raft_consensus.cc:399] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:37.862673 1101 raft_consensus.cc:493] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:37.862805 1101 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:37.864845 1101 raft_consensus.cc:515] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aebce2852341ff8abb004c161dcd5e" member_type: VOTER }
I20251212 08:52:37.865073 1101 leader_election.cc:304] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5aebce2852341ff8abb004c161dcd5e; no voters:
I20251212 08:52:37.865592 1101 leader_election.cc:290] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:37.866288 1104 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:37.866997 1104 raft_consensus.cc:697] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 1 LEADER]: Becoming Leader. State: Replica: d5aebce2852341ff8abb004c161dcd5e, State: Running, Role: LEADER
I20251212 08:52:37.867233 1101 sys_catalog.cc:565] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:37.867319 1104 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aebce2852341ff8abb004c161dcd5e" member_type: VOTER }
I20251212 08:52:37.868644 1105 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5aebce2852341ff8abb004c161dcd5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aebce2852341ff8abb004c161dcd5e" member_type: VOTER } }
I20251212 08:52:37.868958 1105 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:37.869978 1106 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5aebce2852341ff8abb004c161dcd5e. Latest consensus state: current_term: 1 leader_uuid: "d5aebce2852341ff8abb004c161dcd5e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5aebce2852341ff8abb004c161dcd5e" member_type: VOTER } }
I20251212 08:52:37.870292 1106 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:37.876751 1109 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:37.879519 1109 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:37.881418 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:37.883694 1109 catalog_manager.cc:1357] Generated new cluster ID: af1e9c8549bb435dbfabdddd952290a0
I20251212 08:52:37.883817 1109 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:37.902130 1109 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:37.902983 1109 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:37.917657 1109 catalog_manager.cc:6027] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e: Generated new TSK 0
I20251212 08:52:37.917925 1109 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:37.947118 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:37.950352 1123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:37.951093 1124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:37.951467 1126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:37.951756 832 server_base.cc:1047] running on GCE node
I20251212 08:52:37.952036 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:37.952111 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:37.952191 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529557952153 us; error 0 us; skew 500 ppm
I20251212 08:52:37.952389 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:37.953536 832 webserver.cc:492] Webserver started at http://127.0.208.1:36207/ using document root <none> and password file <none>
I20251212 08:52:37.953742 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:37.953845 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:37.953972 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:37.954783 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "ba11642dbddb41499491a9281be91b80"
format_stamp: "Formatted at 2025-12-12 08:52:37 on dist-test-slave-qdpb"
I20251212 08:52:37.957407 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20251212 08:52:37.959020 1131 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:37.959369 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:52:37.959488 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "ba11642dbddb41499491a9281be91b80"
format_stamp: "Formatted at 2025-12-12 08:52:37 on dist-test-slave-qdpb"
I20251212 08:52:37.959623 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScanAtFutureTimestamp.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:37.981184 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:37.981828 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:37.983553 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:37.983682 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:37.983945 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:37.984076 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20251212 08:52:38.013057 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:44715
I20251212 08:52:38.013108 1201 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:44715 every 8 connection(s)
I20251212 08:52:38.015128 1202 heartbeater.cc:344] Connected to a master server at 127.0.208.62:44445
I20251212 08:52:38.015276 1202 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:38.015707 1202 heartbeater.cc:507] Master 127.0.208.62:44445 requested a full tablet report, sending...
I20251212 08:52:38.016636 1056 ts_manager.cc:194] Registered new tserver with Master: ba11642dbddb41499491a9281be91b80 (127.0.208.1:44715)
I20251212 08:52:38.017661 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.003557929s
I20251212 08:52:38.017715 1056 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37084
I20251212 08:52:38.029069 1202 heartbeater.cc:499] Master 127.0.208.62:44445 was elected leader, sending a full tablet report...
I20251212 08:52:38.029371 1056 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37096:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:38.043994 1162 tablet_service.cc:1505] Processing CreateTablet for tablet 6db12a309cb64fd2be4285f44754fcd5 (DEFAULT_TABLE table=client-testtb [id=4d9a6005798b4d9b8ced02baf08d8d72]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:38.043994 1161 tablet_service.cc:1505] Processing CreateTablet for tablet 57f8e278626c4f73af867fc60a5ad985 (DEFAULT_TABLE table=client-testtb [id=4d9a6005798b4d9b8ced02baf08d8d72]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:38.044659 1162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db12a309cb64fd2be4285f44754fcd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.044845 1161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57f8e278626c4f73af867fc60a5ad985. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.051024 1212 tablet_bootstrap.cc:492] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80: Bootstrap starting.
I20251212 08:52:38.052785 1212 tablet_bootstrap.cc:654] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.054967 1212 tablet_bootstrap.cc:492] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80: No bootstrap required, opened a new log
I20251212 08:52:38.055131 1212 ts_tablet_manager.cc:1403] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251212 08:52:38.055814 1212 raft_consensus.cc:359] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } }
I20251212 08:52:38.055985 1212 raft_consensus.cc:385] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.056089 1212 raft_consensus.cc:740] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba11642dbddb41499491a9281be91b80, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.056334 1212 consensus_queue.cc:260] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } }
I20251212 08:52:38.056501 1212 raft_consensus.cc:399] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.056582 1212 raft_consensus.cc:493] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.056699 1212 raft_consensus.cc:3060] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.058557 1212 raft_consensus.cc:515] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } }
I20251212 08:52:38.058770 1212 leader_election.cc:304] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba11642dbddb41499491a9281be91b80; no voters:
I20251212 08:52:38.059195 1212 leader_election.cc:290] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.059377 1214 raft_consensus.cc:2804] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.060134 1212 ts_tablet_manager.cc:1434] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20251212 08:52:38.060504 1212 tablet_bootstrap.cc:492] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80: Bootstrap starting.
I20251212 08:52:38.060954 1214 raft_consensus.cc:697] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 1 LEADER]: Becoming Leader. State: Replica: ba11642dbddb41499491a9281be91b80, State: Running, Role: LEADER
I20251212 08:52:38.061313 1214 consensus_queue.cc:237] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } }
I20251212 08:52:38.062852 1212 tablet_bootstrap.cc:654] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.064805 1212 tablet_bootstrap.cc:492] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80: No bootstrap required, opened a new log
I20251212 08:52:38.064653 1056 catalog_manager.cc:5654] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba11642dbddb41499491a9281be91b80 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "ba11642dbddb41499491a9281be91b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:38.064977 1212 ts_tablet_manager.cc:1403] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251212 08:52:38.065582 1212 raft_consensus.cc:359] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } }
I20251212 08:52:38.065758 1212 raft_consensus.cc:385] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.065850 1212 raft_consensus.cc:740] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba11642dbddb41499491a9281be91b80, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.066092 1212 consensus_queue.cc:260] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } }
I20251212 08:52:38.066231 1212 raft_consensus.cc:399] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.066336 1212 raft_consensus.cc:493] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.066442 1212 raft_consensus.cc:3060] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.069293 1212 raft_consensus.cc:515] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } }
I20251212 08:52:38.069514 1212 leader_election.cc:304] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba11642dbddb41499491a9281be91b80; no voters:
I20251212 08:52:38.069725 1212 leader_election.cc:290] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.069990 1216 raft_consensus.cc:2804] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.070139 1216 raft_consensus.cc:697] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 1 LEADER]: Becoming Leader. State: Replica: ba11642dbddb41499491a9281be91b80, State: Running, Role: LEADER
I20251212 08:52:38.070235 1212 ts_tablet_manager.cc:1434] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80: Time spent starting tablet: real 0.005s user 0.001s sys 0.005s
I20251212 08:52:38.070422 1216 consensus_queue.cc:237] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } }
I20251212 08:52:38.072538 1056 catalog_manager.cc:5654] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 reported cstate change: term changed from 0 to 1, leader changed from <none> to ba11642dbddb41499491a9281be91b80 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "ba11642dbddb41499491a9281be91b80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba11642dbddb41499491a9281be91b80" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 44715 } health_report { overall_health: HEALTHY } } }
W20251212 08:52:38.097823 1152 tablet_service.cc:3074] error setting up scanner db9232988ba8413da5b23382a05114e3 with request: Invalid argument: cannot verify timestamp: Snapshot time P: 1765529618079616 usec, L: 0 in the future. Max allowed timestamp is P: 1765529568097657 usec, L: 0: new_scan_request { tablet_id: "6db12a309cb64fd2be4285f44754fcd5" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } projected_columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" write_default_value: "90\000\000" immutable: false } read_mode: READ_AT_SNAPSHOT snap_timestamp: 7231609315654107136 propagated_timestamp: 7231609069953400832 cache_blocks: true order_mode: UNORDERED row_format_flags: 0 authz_token { token_data: "\010\202\262\357\311\006\"3\n\005slave\022*\n 4d9a6005798b4d9b8ced02baf08d8d72\020\001\030\001 \001(\001" signature: "PvXsg\031\211\215*K}\212\341\343\254\257\352\331H;0\321\325\332\372#\237<[#\257$SlR\212\220.\013+\207\270\"\346\254P\204d\244\025m\357J3}E\353\255ej\ni\301\365""PvXsg\031\211\215*K}\212\341\343\254\257\352\331H;0\321\325\332\372#\237<[#\257$SlR\212\220.\013+\207\270\"\346\254P\204d\244\025m\357J3}E\353\255ej\ni\301\365" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "ef262bcf6100409b9fcdfaf8d5d817e7"
I20251212 08:52:38.100456 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:52:38.110814 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:52:38.111227 832 tablet_replica.cc:333] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80: stopping tablet replica
I20251212 08:52:38.111430 832 raft_consensus.cc:2243] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.111621 832 raft_consensus.cc:2272] T 57f8e278626c4f73af867fc60a5ad985 P ba11642dbddb41499491a9281be91b80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:38.112574 832 tablet_replica.cc:333] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80: stopping tablet replica
I20251212 08:52:38.112747 832 raft_consensus.cc:2243] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.112927 832 raft_consensus.cc:2272] T 6db12a309cb64fd2be4285f44754fcd5 P ba11642dbddb41499491a9281be91b80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:38.127986 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:52:38.132386 832 master.cc:561] Master@127.0.208.62:44445 shutting down...
I20251212 08:52:38.142674 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.142930 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:38.143116 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P d5aebce2852341ff8abb004c161dcd5e: stopping tablet replica
I20251212 08:52:38.157805 832 master.cc:583] Master@127.0.208.62:44445 shutdown complete.
[ OK ] ClientTest.TestScanAtFutureTimestamp (371 ms)
[ RUN ] ClientTest.TestInvalidPredicates
I20251212 08:52:38.169620 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:42785
I20251212 08:52:38.170233 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:38.173062 1224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:38.173388 1223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:38.174326 1226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:38.175225 832 server_base.cc:1047] running on GCE node
I20251212 08:52:38.175570 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:38.175637 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:38.175716 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529558175678 us; error 0 us; skew 500 ppm
I20251212 08:52:38.175915 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:38.177389 832 webserver.cc:492] Webserver started at http://127.0.208.62:39385/ using document root <none> and password file <none>
I20251212 08:52:38.177627 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:38.177723 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:38.177867 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:38.178553 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "4acd31f45a224636921e06c2f5c70ce6"
format_stamp: "Formatted at 2025-12-12 08:52:38 on dist-test-slave-qdpb"
I20251212 08:52:38.181326 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20251212 08:52:38.183244 1231 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:38.183706 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:38.183828 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "4acd31f45a224636921e06c2f5c70ce6"
format_stamp: "Formatted at 2025-12-12 08:52:38 on dist-test-slave-qdpb"
I20251212 08:52:38.183986 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:38.196210 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:38.196926 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:38.216149 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:42785
I20251212 08:52:38.216162 1292 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:42785 every 8 connection(s)
I20251212 08:52:38.217737 1293 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.224589 1293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6: Bootstrap starting.
I20251212 08:52:38.226634 1293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.229350 1293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6: No bootstrap required, opened a new log
I20251212 08:52:38.230252 1293 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acd31f45a224636921e06c2f5c70ce6" member_type: VOTER }
I20251212 08:52:38.230439 1293 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.230536 1293 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acd31f45a224636921e06c2f5c70ce6, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.230778 1293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acd31f45a224636921e06c2f5c70ce6" member_type: VOTER }
I20251212 08:52:38.230948 1293 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.231033 1293 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.231146 1293 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.234274 1293 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acd31f45a224636921e06c2f5c70ce6" member_type: VOTER }
I20251212 08:52:38.234544 1293 leader_election.cc:304] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4acd31f45a224636921e06c2f5c70ce6; no voters:
I20251212 08:52:38.235008 1293 leader_election.cc:290] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.235170 1296 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.235934 1296 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 1 LEADER]: Becoming Leader. State: Replica: 4acd31f45a224636921e06c2f5c70ce6, State: Running, Role: LEADER
I20251212 08:52:38.236043 1293 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:38.236243 1296 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acd31f45a224636921e06c2f5c70ce6" member_type: VOTER }
I20251212 08:52:38.237273 1297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4acd31f45a224636921e06c2f5c70ce6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acd31f45a224636921e06c2f5c70ce6" member_type: VOTER } }
I20251212 08:52:38.237491 1297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:38.243175 1301 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:38.243317 1298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acd31f45a224636921e06c2f5c70ce6. Latest consensus state: current_term: 1 leader_uuid: "4acd31f45a224636921e06c2f5c70ce6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4acd31f45a224636921e06c2f5c70ce6" member_type: VOTER } }
I20251212 08:52:38.244051 1298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:38.246047 1301 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:38.248049 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:38.249945 1301 catalog_manager.cc:1357] Generated new cluster ID: 949b27d9ac0347549a9d2f472eba070a
I20251212 08:52:38.250044 1301 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:38.267699 1301 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:38.268923 1301 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:38.278393 1301 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6: Generated new TSK 0
I20251212 08:52:38.278687 1301 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:38.280862 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:38.284049 1316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:38.284050 1315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:38.284951 1318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:38.285740 832 server_base.cc:1047] running on GCE node
I20251212 08:52:38.286583 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:38.286700 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:38.286764 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529558286743 us; error 0 us; skew 500 ppm
I20251212 08:52:38.286960 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:38.288394 832 webserver.cc:492] Webserver started at http://127.0.208.1:33827/ using document root <none> and password file <none>
I20251212 08:52:38.288687 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:38.288784 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:38.288959 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:38.289806 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "0b3a375aa2904cecb17636db9f6488de"
format_stamp: "Formatted at 2025-12-12 08:52:38 on dist-test-slave-qdpb"
I20251212 08:52:38.293143 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20251212 08:52:38.295363 1323 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:38.295852 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:38.295974 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "0b3a375aa2904cecb17636db9f6488de"
format_stamp: "Formatted at 2025-12-12 08:52:38 on dist-test-slave-qdpb"
I20251212 08:52:38.296226 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInvalidPredicates.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:38.316641 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:38.317353 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:38.319443 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:38.319561 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:38.320176 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:38.320264 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20251212 08:52:38.375639 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:42859
I20251212 08:52:38.375662 1393 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:42859 every 8 connection(s)
I20251212 08:52:38.378423 1394 heartbeater.cc:344] Connected to a master server at 127.0.208.62:42785
I20251212 08:52:38.378566 1394 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:38.379046 1394 heartbeater.cc:507] Master 127.0.208.62:42785 requested a full tablet report, sending...
I20251212 08:52:38.380059 1248 ts_manager.cc:194] Registered new tserver with Master: 0b3a375aa2904cecb17636db9f6488de (127.0.208.1:42859)
I20251212 08:52:38.380966 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.003505225s
I20251212 08:52:38.381760 1248 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49102
I20251212 08:52:38.393476 1394 heartbeater.cc:499] Master 127.0.208.62:42785 was elected leader, sending a full tablet report...
I20251212 08:52:38.395284 1248 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49114:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:38.419023 1355 tablet_service.cc:1505] Processing CreateTablet for tablet ef2503ba67e34ca0a6e741268703aaa4 (DEFAULT_TABLE table=client-testtb [id=04fe555a3e4c4e7e903e5d8f213023f5]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:38.419700 1355 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef2503ba67e34ca0a6e741268703aaa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.420146 1354 tablet_service.cc:1505] Processing CreateTablet for tablet 1d7482c8e882456eb6633f0c2e6c0982 (DEFAULT_TABLE table=client-testtb [id=04fe555a3e4c4e7e903e5d8f213023f5]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:38.420754 1354 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1d7482c8e882456eb6633f0c2e6c0982. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.426676 1404 tablet_bootstrap.cc:492] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de: Bootstrap starting.
I20251212 08:52:38.428610 1404 tablet_bootstrap.cc:654] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.433519 1404 tablet_bootstrap.cc:492] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de: No bootstrap required, opened a new log
I20251212 08:52:38.433670 1404 ts_tablet_manager.cc:1403] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20251212 08:52:38.434329 1404 raft_consensus.cc:359] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } }
I20251212 08:52:38.434501 1404 raft_consensus.cc:385] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.434603 1404 raft_consensus.cc:740] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b3a375aa2904cecb17636db9f6488de, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.434839 1404 consensus_queue.cc:260] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } }
I20251212 08:52:38.435002 1404 raft_consensus.cc:399] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.435098 1404 raft_consensus.cc:493] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.435230 1404 raft_consensus.cc:3060] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.438202 1404 raft_consensus.cc:515] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } }
I20251212 08:52:38.438436 1404 leader_election.cc:304] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b3a375aa2904cecb17636db9f6488de; no voters:
I20251212 08:52:38.438959 1404 leader_election.cc:290] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.439126 1406 raft_consensus.cc:2804] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.439764 1406 raft_consensus.cc:697] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 1 LEADER]: Becoming Leader. State: Replica: 0b3a375aa2904cecb17636db9f6488de, State: Running, Role: LEADER
I20251212 08:52:38.440228 1406 consensus_queue.cc:237] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } }
I20251212 08:52:38.443436 1248 catalog_manager.cc:5654] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b3a375aa2904cecb17636db9f6488de (127.0.208.1). New cstate: current_term: 1 leader_uuid: "0b3a375aa2904cecb17636db9f6488de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:38.449621 1404 ts_tablet_manager.cc:1434] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de: Time spent starting tablet: real 0.016s user 0.005s sys 0.011s
I20251212 08:52:38.449925 1404 tablet_bootstrap.cc:492] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de: Bootstrap starting.
I20251212 08:52:38.451375 1404 tablet_bootstrap.cc:654] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.453837 1404 tablet_bootstrap.cc:492] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de: No bootstrap required, opened a new log
I20251212 08:52:38.454005 1404 ts_tablet_manager.cc:1403] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:38.454635 1404 raft_consensus.cc:359] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } }
I20251212 08:52:38.454854 1404 raft_consensus.cc:385] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.454958 1404 raft_consensus.cc:740] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b3a375aa2904cecb17636db9f6488de, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.455302 1404 consensus_queue.cc:260] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } }
I20251212 08:52:38.455571 1404 raft_consensus.cc:399] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.455705 1404 raft_consensus.cc:493] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.455854 1404 raft_consensus.cc:3060] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.459002 1404 raft_consensus.cc:515] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } }
I20251212 08:52:38.459224 1404 leader_election.cc:304] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0b3a375aa2904cecb17636db9f6488de; no voters:
I20251212 08:52:38.459636 1404 leader_election.cc:290] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.459832 1408 raft_consensus.cc:2804] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.460394 1404 ts_tablet_manager.cc:1434] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20251212 08:52:38.460633 1408 raft_consensus.cc:697] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 1 LEADER]: Becoming Leader. State: Replica: 0b3a375aa2904cecb17636db9f6488de, State: Running, Role: LEADER
I20251212 08:52:38.460943 1408 consensus_queue.cc:237] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } }
I20251212 08:52:38.463304 1248 catalog_manager.cc:5654] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b3a375aa2904cecb17636db9f6488de (127.0.208.1). New cstate: current_term: 1 leader_uuid: "0b3a375aa2904cecb17636db9f6488de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0b3a375aa2904cecb17636db9f6488de" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 42859 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:38.473187 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:52:38.483176 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:52:38.483623 832 tablet_replica.cc:333] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de: stopping tablet replica
I20251212 08:52:38.483812 832 raft_consensus.cc:2243] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.484158 832 raft_consensus.cc:2272] T ef2503ba67e34ca0a6e741268703aaa4 P 0b3a375aa2904cecb17636db9f6488de [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:38.485394 832 tablet_replica.cc:333] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de: stopping tablet replica
I20251212 08:52:38.485638 832 raft_consensus.cc:2243] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.485809 832 raft_consensus.cc:2272] T 1d7482c8e882456eb6633f0c2e6c0982 P 0b3a375aa2904cecb17636db9f6488de [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:38.491217 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:52:38.495676 832 master.cc:561] Master@127.0.208.62:42785 shutting down...
I20251212 08:52:38.506826 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.507103 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:38.507334 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4acd31f45a224636921e06c2f5c70ce6: stopping tablet replica
I20251212 08:52:38.522302 832 master.cc:583] Master@127.0.208.62:42785 shutdown complete.
[ OK ] ClientTest.TestInvalidPredicates (363 ms)
[ RUN ] ClientTest.TestExclusiveInclusiveRangeBounds
I20251212 08:52:38.533420 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:40355
I20251212 08:52:38.534026 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:38.536806 1413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:38.537328 1414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:38.539319 1416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:38.539455 832 server_base.cc:1047] running on GCE node
I20251212 08:52:38.539875 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:38.539954 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:38.540026 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529558539999 us; error 0 us; skew 500 ppm
I20251212 08:52:38.540230 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:38.541620 832 webserver.cc:492] Webserver started at http://127.0.208.62:37535/ using document root <none> and password file <none>
I20251212 08:52:38.541851 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:38.541940 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:38.542119 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:38.542770 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "e810458865944bc39836b8a95a027ea2"
format_stamp: "Formatted at 2025-12-12 08:52:38 on dist-test-slave-qdpb"
I20251212 08:52:38.545557 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20251212 08:52:38.547293 1421 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:38.547688 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:38.547812 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "e810458865944bc39836b8a95a027ea2"
format_stamp: "Formatted at 2025-12-12 08:52:38 on dist-test-slave-qdpb"
I20251212 08:52:38.547945 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:38.556459 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:38.557175 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:38.579641 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:40355
I20251212 08:52:38.579692 1482 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:40355 every 8 connection(s)
I20251212 08:52:38.581382 1483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.588276 1483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2: Bootstrap starting.
I20251212 08:52:38.591068 1483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.593498 1483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2: No bootstrap required, opened a new log
I20251212 08:52:38.594448 1483 raft_consensus.cc:359] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e810458865944bc39836b8a95a027ea2" member_type: VOTER }
I20251212 08:52:38.594651 1483 raft_consensus.cc:385] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.594758 1483 raft_consensus.cc:740] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e810458865944bc39836b8a95a027ea2, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.595023 1483 consensus_queue.cc:260] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e810458865944bc39836b8a95a027ea2" member_type: VOTER }
I20251212 08:52:38.595201 1483 raft_consensus.cc:399] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.595297 1483 raft_consensus.cc:493] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.595428 1483 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.598642 1483 raft_consensus.cc:515] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e810458865944bc39836b8a95a027ea2" member_type: VOTER }
I20251212 08:52:38.598942 1483 leader_election.cc:304] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e810458865944bc39836b8a95a027ea2; no voters:
I20251212 08:52:38.599545 1483 leader_election.cc:290] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.599749 1486 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.600557 1486 raft_consensus.cc:697] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 1 LEADER]: Becoming Leader. State: Replica: e810458865944bc39836b8a95a027ea2, State: Running, Role: LEADER
I20251212 08:52:38.600752 1483 sys_catalog.cc:565] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:38.600901 1486 consensus_queue.cc:237] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e810458865944bc39836b8a95a027ea2" member_type: VOTER }
I20251212 08:52:38.602420 1487 sys_catalog.cc:455] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e810458865944bc39836b8a95a027ea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e810458865944bc39836b8a95a027ea2" member_type: VOTER } }
I20251212 08:52:38.602705 1487 sys_catalog.cc:458] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:38.602967 1488 sys_catalog.cc:455] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e810458865944bc39836b8a95a027ea2. Latest consensus state: current_term: 1 leader_uuid: "e810458865944bc39836b8a95a027ea2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e810458865944bc39836b8a95a027ea2" member_type: VOTER } }
I20251212 08:52:38.603153 1488 sys_catalog.cc:458] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:38.604132 1490 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:38.606128 1490 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:38.611406 1490 catalog_manager.cc:1357] Generated new cluster ID: 8c4864abfc4145eea6609dea8055da20
I20251212 08:52:38.611560 1490 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:38.616398 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:38.636413 1490 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:38.637324 1490 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:38.647935 1490 catalog_manager.cc:6027] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2: Generated new TSK 0
I20251212 08:52:38.648236 1490 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:38.681372 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:38.685369 1504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:38.686446 1505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:38.686882 1507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:38.687069 832 server_base.cc:1047] running on GCE node
I20251212 08:52:38.687557 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:38.687642 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:38.687708 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529558687671 us; error 0 us; skew 500 ppm
I20251212 08:52:38.687877 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:38.689268 832 webserver.cc:492] Webserver started at http://127.0.208.1:44643/ using document root <none> and password file <none>
I20251212 08:52:38.689486 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:38.689586 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:38.689738 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:38.690402 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "227d88789c6645ae99688ab00034705a"
format_stamp: "Formatted at 2025-12-12 08:52:38 on dist-test-slave-qdpb"
I20251212 08:52:38.693040 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:52:38.694696 1512 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:38.695097 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:52:38.695199 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "227d88789c6645ae99688ab00034705a"
format_stamp: "Formatted at 2025-12-12 08:52:38 on dist-test-slave-qdpb"
I20251212 08:52:38.695344 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:38.703775 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:38.704409 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:38.706009 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:38.706151 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:38.706288 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:38.706405 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:38.730928 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35105
I20251212 08:52:38.730947 1582 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35105 every 8 connection(s)
I20251212 08:52:38.732839 1583 heartbeater.cc:344] Connected to a master server at 127.0.208.62:40355
I20251212 08:52:38.732980 1583 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:38.733469 1583 heartbeater.cc:507] Master 127.0.208.62:40355 requested a full tablet report, sending...
I20251212 08:52:38.734215 1438 ts_manager.cc:194] Registered new tserver with Master: 227d88789c6645ae99688ab00034705a (127.0.208.1:35105)
I20251212 08:52:38.734580 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002314602s
I20251212 08:52:38.735704 1438 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37776
I20251212 08:52:38.746621 1438 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37794:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:38.747151 1583 heartbeater.cc:499] Master 127.0.208.62:40355 was elected leader, sending a full tablet report...
I20251212 08:52:38.764362 1542 tablet_service.cc:1505] Processing CreateTablet for tablet 426b63c227214862bb311fda7b7512e7 (DEFAULT_TABLE table=client-testtb [id=6dd48c92cd4f43eb8aa7b9b497f6e512]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:38.764353 1543 tablet_service.cc:1505] Processing CreateTablet for tablet fc9e44c7e60a45d39095070b36dc63f1 (DEFAULT_TABLE table=client-testtb [id=6dd48c92cd4f43eb8aa7b9b497f6e512]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:38.764969 1542 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 426b63c227214862bb311fda7b7512e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.765220 1543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc9e44c7e60a45d39095070b36dc63f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.770838 1593 tablet_bootstrap.cc:492] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a: Bootstrap starting.
I20251212 08:52:38.773326 1593 tablet_bootstrap.cc:654] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.775019 1593 tablet_bootstrap.cc:492] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a: No bootstrap required, opened a new log
I20251212 08:52:38.775153 1593 ts_tablet_manager.cc:1403] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20251212 08:52:38.775791 1593 raft_consensus.cc:359] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.775959 1593 raft_consensus.cc:385] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.776032 1593 raft_consensus.cc:740] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227d88789c6645ae99688ab00034705a, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.776233 1593 consensus_queue.cc:260] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.776445 1593 raft_consensus.cc:399] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.776510 1593 raft_consensus.cc:493] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.776597 1593 raft_consensus.cc:3060] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.778720 1593 raft_consensus.cc:515] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.778923 1593 leader_election.cc:304] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 227d88789c6645ae99688ab00034705a; no voters:
I20251212 08:52:38.779330 1593 leader_election.cc:290] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.779517 1595 raft_consensus.cc:2804] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.780072 1593 ts_tablet_manager.cc:1434] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20251212 08:52:38.780400 1593 tablet_bootstrap.cc:492] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a: Bootstrap starting.
I20251212 08:52:38.780472 1595 raft_consensus.cc:697] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 1 LEADER]: Becoming Leader. State: Replica: 227d88789c6645ae99688ab00034705a, State: Running, Role: LEADER
I20251212 08:52:38.780790 1595 consensus_queue.cc:237] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.782856 1593 tablet_bootstrap.cc:654] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.783605 1438 catalog_manager.cc:5654] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a reported cstate change: term changed from 0 to 1, leader changed from <none> to 227d88789c6645ae99688ab00034705a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "227d88789c6645ae99688ab00034705a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:38.785113 1593 tablet_bootstrap.cc:492] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a: No bootstrap required, opened a new log
I20251212 08:52:38.785305 1593 ts_tablet_manager.cc:1403] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251212 08:52:38.786165 1593 raft_consensus.cc:359] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.786365 1593 raft_consensus.cc:385] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.786489 1593 raft_consensus.cc:740] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227d88789c6645ae99688ab00034705a, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.786798 1593 consensus_queue.cc:260] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.786940 1593 raft_consensus.cc:399] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.787051 1593 raft_consensus.cc:493] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.787160 1593 raft_consensus.cc:3060] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.789434 1593 raft_consensus.cc:515] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.789634 1593 leader_election.cc:304] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 227d88789c6645ae99688ab00034705a; no voters:
I20251212 08:52:38.789848 1593 leader_election.cc:290] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.789899 1595 raft_consensus.cc:2804] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.790056 1595 raft_consensus.cc:697] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 1 LEADER]: Becoming Leader. State: Replica: 227d88789c6645ae99688ab00034705a, State: Running, Role: LEADER
I20251212 08:52:38.790326 1593 ts_tablet_manager.cc:1434] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20251212 08:52:38.790324 1595 consensus_queue.cc:237] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.792393 1438 catalog_manager.cc:5654] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a reported cstate change: term changed from 0 to 1, leader changed from <none> to 227d88789c6645ae99688ab00034705a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "227d88789c6645ae99688ab00034705a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:38.807814 1438 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37794:
name: "TestExclusiveInclusiveRangeBounds"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\010\001\000\377\377\377\377\t\001\000c\000\000\000""\010\001\000\377\377\377\377\t\001\000c\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:38.815555 1543 tablet_service.cc:1505] Processing CreateTablet for tablet 41b0ec80952a4bf88b8520062cdccdbe (DEFAULT_TABLE table=TestExclusiveInclusiveRangeBounds [id=20ef59750e814c4cb64354ac0905d4ff]), partition=RANGE (key) PARTITION 0 <= VALUES < 100
I20251212 08:52:38.816074 1543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41b0ec80952a4bf88b8520062cdccdbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.821636 1593 tablet_bootstrap.cc:492] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a: Bootstrap starting.
I20251212 08:52:38.823257 1593 tablet_bootstrap.cc:654] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.825323 1593 tablet_bootstrap.cc:492] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a: No bootstrap required, opened a new log
I20251212 08:52:38.825472 1593 ts_tablet_manager.cc:1403] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:52:38.826149 1593 raft_consensus.cc:359] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.826303 1593 raft_consensus.cc:385] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.826402 1593 raft_consensus.cc:740] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227d88789c6645ae99688ab00034705a, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.826609 1593 consensus_queue.cc:260] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.826751 1593 raft_consensus.cc:399] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.826828 1593 raft_consensus.cc:493] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.826941 1593 raft_consensus.cc:3060] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.829002 1593 raft_consensus.cc:515] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.829280 1593 leader_election.cc:304] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 227d88789c6645ae99688ab00034705a; no voters:
I20251212 08:52:38.829523 1593 leader_election.cc:290] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.829614 1595 raft_consensus.cc:2804] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.829841 1595 raft_consensus.cc:697] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 1 LEADER]: Becoming Leader. State: Replica: 227d88789c6645ae99688ab00034705a, State: Running, Role: LEADER
I20251212 08:52:38.830035 1593 ts_tablet_manager.cc:1434] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20251212 08:52:38.830120 1595 consensus_queue.cc:237] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.832326 1438 catalog_manager.cc:5654] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a reported cstate change: term changed from 0 to 1, leader changed from <none> to 227d88789c6645ae99688ab00034705a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "227d88789c6645ae99688ab00034705a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:38.841874 1438 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37794:
table { table_name: "TestExclusiveInclusiveRangeBounds" } alter_schema_steps { type: ADD_RANGE_PARTITION add_range_partition { range_bounds { rows: "\010\001\000\307\000\000\000\t\001\000+\001\000\000""\010\001\000\307\000\000\000\t\001\000+\001\000\000" indirect_data: """" } } } schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } modify_external_catalogs: true
W20251212 08:52:38.846547 1438 catalog_manager.cc:5914] Failed to send alter table request: Not found: failed to reset TS proxy: No leader found: Tablet 4bfb9dfaabf74106befa347929e79139 has no consensus state
I20251212 08:52:38.850768 1543 tablet_service.cc:1505] Processing CreateTablet for tablet 4bfb9dfaabf74106befa347929e79139 (DEFAULT_TABLE table=TestExclusiveInclusiveRangeBounds [id=20ef59750e814c4cb64354ac0905d4ff]), partition=RANGE (key) PARTITION 200 <= VALUES < 300
I20251212 08:52:38.851208 1543 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bfb9dfaabf74106befa347929e79139. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:38.855747 1593 tablet_bootstrap.cc:492] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: Bootstrap starting.
I20251212 08:52:38.857429 1593 tablet_bootstrap.cc:654] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:38.859242 1593 tablet_bootstrap.cc:492] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: No bootstrap required, opened a new log
I20251212 08:52:38.859422 1593 ts_tablet_manager.cc:1403] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:52:38.860229 1593 raft_consensus.cc:359] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.860464 1593 raft_consensus.cc:385] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:38.860549 1593 raft_consensus.cc:740] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 227d88789c6645ae99688ab00034705a, State: Initialized, Role: FOLLOWER
I20251212 08:52:38.860733 1593 consensus_queue.cc:260] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.860843 1593 raft_consensus.cc:399] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:38.860911 1593 raft_consensus.cc:493] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:38.860977 1593 raft_consensus.cc:3060] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:38.863039 1593 raft_consensus.cc:515] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.863256 1593 leader_election.cc:304] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 227d88789c6645ae99688ab00034705a; no voters:
I20251212 08:52:38.863476 1593 leader_election.cc:290] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:38.863555 1595 raft_consensus.cc:2804] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:38.863744 1595 raft_consensus.cc:697] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 1 LEADER]: Becoming Leader. State: Replica: 227d88789c6645ae99688ab00034705a, State: Running, Role: LEADER
I20251212 08:52:38.863948 1593 ts_tablet_manager.cc:1434] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20251212 08:52:38.863987 1595 consensus_queue.cc:237] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } }
I20251212 08:52:38.866557 1437 catalog_manager.cc:5654] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a reported cstate change: term changed from 0 to 1, leader changed from <none> to 227d88789c6645ae99688ab00034705a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "227d88789c6645ae99688ab00034705a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "227d88789c6645ae99688ab00034705a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35105 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:38.870954 1437 catalog_manager.cc:6199] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2: TestExclusiveInclusiveRangeBounds [id=20ef59750e814c4cb64354ac0905d4ff] alter complete (version 0)
I20251212 08:52:38.928643 1437 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:37794:
table { table_name: "TestExclusiveInclusiveRangeBounds" } alter_schema_steps { type: DROP_RANGE_PARTITION drop_range_partition { range_bounds { rows: "\006\001\000\000\000\000\000\007\001\000d\000\000\000""\006\001\000\000\000\000\000\007\001\000d\000\000\000" indirect_data: """" } } } alter_schema_steps { type: DROP_RANGE_PARTITION drop_range_partition { range_bounds { rows: "\010\001\000\307\000\000\000\t\001\000+\001\000\000""\010\001\000\307\000\000\000\t\001\000+\001\000\000" indirect_data: """" } } } schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } columns { name: "non_null_with_default" type: INT32 is_key: false is_nullable: false read_default_value: "90\000\000" encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } modify_external_catalogs: true
I20251212 08:52:38.932911 1437 catalog_manager.cc:5941] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2: Sending DeleteTablet for 1 replicas of tablet 41b0ec80952a4bf88b8520062cdccdbe
I20251212 08:52:38.933367 1437 catalog_manager.cc:5941] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2: Sending DeleteTablet for 1 replicas of tablet 4bfb9dfaabf74106befa347929e79139
I20251212 08:52:38.933954 1543 tablet_service.cc:1552] Processing DeleteTablet for tablet 41b0ec80952a4bf88b8520062cdccdbe with delete_type TABLET_DATA_DELETED (Partition dropped at 2025-12-12 08:52:38 UTC) from {username='slave'} at 127.0.0.1:37326
I20251212 08:52:38.934044 1542 tablet_service.cc:1552] Processing DeleteTablet for tablet 4bfb9dfaabf74106befa347929e79139 with delete_type TABLET_DATA_DELETED (Partition dropped at 2025-12-12 08:52:38 UTC) from {username='slave'} at 127.0.0.1:37326
I20251212 08:52:38.935173 1608 tablet_replica.cc:333] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: stopping tablet replica
I20251212 08:52:38.935379 1608 raft_consensus.cc:2243] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.935657 1608 raft_consensus.cc:2272] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:38.936723 1608 ts_tablet_manager.cc:1916] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:52:38.938341 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:52:38.947518 1608 ts_tablet_manager.cc:1929] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251212 08:52:38.947739 1608 log.cc:1199] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestExclusiveInclusiveRangeBounds.1765529557162900-832-0/minicluster-data/ts-0-root/wals/4bfb9dfaabf74106befa347929e79139
I20251212 08:52:38.948220 1608 ts_tablet_manager.cc:1950] T 4bfb9dfaabf74106befa347929e79139 P 227d88789c6645ae99688ab00034705a: Deleting consensus metadata
I20251212 08:52:38.948335 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20251212 08:52:38.949251 1422 catalog_manager.cc:4977] TS 227d88789c6645ae99688ab00034705a (127.0.208.1:35105): delete failed for tablet 4bfb9dfaabf74106befa347929e79139 with error code TABLET_NOT_RUNNING: Service unavailable: Tablet Manager is not running: MANAGER_QUIESCING
I20251212 08:52:38.949661 832 tablet_replica.cc:333] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a: stopping tablet replica
W20251212 08:52:38.949654 1422 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: Tablet server shutting down
I20251212 08:52:38.949887 832 raft_consensus.cc:2243] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.950148 832 raft_consensus.cc:2272] T 41b0ec80952a4bf88b8520062cdccdbe P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Raft consensus is shut down!
W20251212 08:52:38.950960 1422 catalog_manager.cc:4712] TS 227d88789c6645ae99688ab00034705a (127.0.208.1:35105): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 41b0ec80952a4bf88b8520062cdccdbe: Remote error: Service unavailable: service kudu.tserver.TabletServerAdminService not registered on TabletServer
I20251212 08:52:38.951143 832 tablet_replica.cc:333] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a: stopping tablet replica
I20251212 08:52:38.951314 832 raft_consensus.cc:2243] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.951467 832 raft_consensus.cc:2272] T fc9e44c7e60a45d39095070b36dc63f1 P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:38.952247 832 tablet_replica.cc:333] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a: stopping tablet replica
I20251212 08:52:38.952422 832 raft_consensus.cc:2243] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:38.952579 832 raft_consensus.cc:2272] T 426b63c227214862bb311fda7b7512e7 P 227d88789c6645ae99688ab00034705a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:38.967393 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:52:38.972651 832 master.cc:561] Master@127.0.208.62:40355 shutting down...
I20251212 08:52:39.008517 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:39.008806 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:39.009034 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P e810458865944bc39836b8a95a027ea2: stopping tablet replica
I20251212 08:52:39.013777 832 master.cc:583] Master@127.0.208.62:40355 shutdown complete.
[ OK ] ClientTest.TestExclusiveInclusiveRangeBounds (490 ms)
[ RUN ] ClientTest.TestBasicIdBasedLookup
I20251212 08:52:39.023833 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:46479
I20251212 08:52:39.024400 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:39.026836 1610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:39.026996 1611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:39.027401 1613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:39.027769 832 server_base.cc:1047] running on GCE node
I20251212 08:52:39.028071 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:39.028136 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:39.028209 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529559028177 us; error 0 us; skew 500 ppm
I20251212 08:52:39.028434 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:39.029647 832 webserver.cc:492] Webserver started at http://127.0.208.62:33545/ using document root <none> and password file <none>
I20251212 08:52:39.029865 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:39.029963 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:39.030115 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:39.030743 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "b898e502592d488cb990963c000cc14d"
format_stamp: "Formatted at 2025-12-12 08:52:39 on dist-test-slave-qdpb"
I20251212 08:52:39.033352 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:52:39.034967 1618 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:39.035321 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:52:39.035447 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "b898e502592d488cb990963c000cc14d"
format_stamp: "Formatted at 2025-12-12 08:52:39 on dist-test-slave-qdpb"
I20251212 08:52:39.035575 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:39.043322 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:39.043893 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:39.059276 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:46479
I20251212 08:52:39.059306 1679 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:46479 every 8 connection(s)
I20251212 08:52:39.060552 1680 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:39.064524 1680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d: Bootstrap starting.
I20251212 08:52:39.065778 1680 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:39.067327 1680 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d: No bootstrap required, opened a new log
I20251212 08:52:39.067893 1680 raft_consensus.cc:359] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b898e502592d488cb990963c000cc14d" member_type: VOTER }
I20251212 08:52:39.068015 1680 raft_consensus.cc:385] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:39.068073 1680 raft_consensus.cc:740] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b898e502592d488cb990963c000cc14d, State: Initialized, Role: FOLLOWER
I20251212 08:52:39.068305 1680 consensus_queue.cc:260] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b898e502592d488cb990963c000cc14d" member_type: VOTER }
I20251212 08:52:39.068447 1680 raft_consensus.cc:399] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:39.068503 1680 raft_consensus.cc:493] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:39.068580 1680 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:39.070231 1680 raft_consensus.cc:515] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b898e502592d488cb990963c000cc14d" member_type: VOTER }
I20251212 08:52:39.070415 1680 leader_election.cc:304] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b898e502592d488cb990963c000cc14d; no voters:
I20251212 08:52:39.070736 1680 leader_election.cc:290] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:39.070897 1683 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:39.071370 1683 raft_consensus.cc:697] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 1 LEADER]: Becoming Leader. State: Replica: b898e502592d488cb990963c000cc14d, State: Running, Role: LEADER
I20251212 08:52:39.071635 1683 consensus_queue.cc:237] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b898e502592d488cb990963c000cc14d" member_type: VOTER }
I20251212 08:52:39.071683 1680 sys_catalog.cc:565] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:39.072540 1684 sys_catalog.cc:455] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b898e502592d488cb990963c000cc14d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b898e502592d488cb990963c000cc14d" member_type: VOTER } }
I20251212 08:52:39.072613 1685 sys_catalog.cc:455] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b898e502592d488cb990963c000cc14d. Latest consensus state: current_term: 1 leader_uuid: "b898e502592d488cb990963c000cc14d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b898e502592d488cb990963c000cc14d" member_type: VOTER } }
I20251212 08:52:39.072918 1685 sys_catalog.cc:458] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:39.072955 1684 sys_catalog.cc:458] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:39.073809 1688 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:39.075268 1688 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:39.077109 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:39.078501 1688 catalog_manager.cc:1357] Generated new cluster ID: 3240a85e22b143c29d3d808bbe4e0f26
I20251212 08:52:39.078574 1688 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:39.088732 1688 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:39.089501 1688 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:39.100566 1688 catalog_manager.cc:6027] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d: Generated new TSK 0
I20251212 08:52:39.100852 1688 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:39.109792 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:39.112665 1701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:39.112735 1702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:39.113358 1704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:39.114032 832 server_base.cc:1047] running on GCE node
I20251212 08:52:39.114248 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:39.114320 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:39.114362 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529559114349 us; error 0 us; skew 500 ppm
I20251212 08:52:39.114485 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:39.115566 832 webserver.cc:492] Webserver started at http://127.0.208.1:35867/ using document root <none> and password file <none>
I20251212 08:52:39.115794 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:39.115861 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:39.115976 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:39.116616 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "511fc2eb89c64ea7819656f6f1412221"
format_stamp: "Formatted at 2025-12-12 08:52:39 on dist-test-slave-qdpb"
I20251212 08:52:39.118934 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:39.120577 1709 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:39.121464 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:39.121568 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "511fc2eb89c64ea7819656f6f1412221"
format_stamp: "Formatted at 2025-12-12 08:52:39 on dist-test-slave-qdpb"
I20251212 08:52:39.121675 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBasicIdBasedLookup.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:39.145233 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:39.145788 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:39.147502 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:39.147610 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:39.147817 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:39.148263 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20251212 08:52:39.171744 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:46651
I20251212 08:52:39.171803 1779 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:46651 every 8 connection(s)
I20251212 08:52:39.173707 1780 heartbeater.cc:344] Connected to a master server at 127.0.208.62:46479
I20251212 08:52:39.173861 1780 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:39.174311 1780 heartbeater.cc:507] Master 127.0.208.62:46479 requested a full tablet report, sending...
I20251212 08:52:39.175240 1635 ts_manager.cc:194] Registered new tserver with Master: 511fc2eb89c64ea7819656f6f1412221 (127.0.208.1:46651)
I20251212 08:52:39.175309 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002423107s
I20251212 08:52:39.176586 1635 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58692
I20251212 08:52:39.187161 1635 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58706:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:39.188285 1780 heartbeater.cc:499] Master 127.0.208.62:46479 was elected leader, sending a full tablet report...
I20251212 08:52:39.201669 1739 tablet_service.cc:1505] Processing CreateTablet for tablet 6ccdbdd5ecdb443eaef05f9b59595dae (DEFAULT_TABLE table=client-testtb [id=cb7f8d529d6c4954aae166a85780082e]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:39.201689 1740 tablet_service.cc:1505] Processing CreateTablet for tablet 81e281d7e3564cada927454855df539f (DEFAULT_TABLE table=client-testtb [id=cb7f8d529d6c4954aae166a85780082e]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:39.202241 1739 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6ccdbdd5ecdb443eaef05f9b59595dae. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:39.202486 1740 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 81e281d7e3564cada927454855df539f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:39.207993 1790 tablet_bootstrap.cc:492] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221: Bootstrap starting.
I20251212 08:52:39.209702 1790 tablet_bootstrap.cc:654] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:39.211448 1790 tablet_bootstrap.cc:492] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221: No bootstrap required, opened a new log
I20251212 08:52:39.211571 1790 ts_tablet_manager.cc:1403] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251212 08:52:39.212168 1790 raft_consensus.cc:359] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } }
I20251212 08:52:39.212359 1790 raft_consensus.cc:385] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:39.212457 1790 raft_consensus.cc:740] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 511fc2eb89c64ea7819656f6f1412221, State: Initialized, Role: FOLLOWER
I20251212 08:52:39.212663 1790 consensus_queue.cc:260] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } }
I20251212 08:52:39.212798 1790 raft_consensus.cc:399] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:39.212880 1790 raft_consensus.cc:493] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:39.212994 1790 raft_consensus.cc:3060] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:39.215049 1790 raft_consensus.cc:515] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } }
I20251212 08:52:39.215262 1790 leader_election.cc:304] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 511fc2eb89c64ea7819656f6f1412221; no voters:
I20251212 08:52:39.215894 1790 leader_election.cc:290] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:39.216683 1792 raft_consensus.cc:2804] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:39.217293 1792 raft_consensus.cc:697] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 1 LEADER]: Becoming Leader. State: Replica: 511fc2eb89c64ea7819656f6f1412221, State: Running, Role: LEADER
I20251212 08:52:39.217384 1790 ts_tablet_manager.cc:1434] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20251212 08:52:39.217702 1790 tablet_bootstrap.cc:492] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221: Bootstrap starting.
I20251212 08:52:39.217628 1792 consensus_queue.cc:237] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } }
I20251212 08:52:39.219635 1790 tablet_bootstrap.cc:654] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:39.220052 1635 catalog_manager.cc:5654] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 reported cstate change: term changed from 0 to 1, leader changed from <none> to 511fc2eb89c64ea7819656f6f1412221 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "511fc2eb89c64ea7819656f6f1412221" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:39.222347 1790 tablet_bootstrap.cc:492] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221: No bootstrap required, opened a new log
I20251212 08:52:39.222501 1790 ts_tablet_manager.cc:1403] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251212 08:52:39.223337 1790 raft_consensus.cc:359] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } }
I20251212 08:52:39.223520 1790 raft_consensus.cc:385] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:39.223608 1790 raft_consensus.cc:740] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 511fc2eb89c64ea7819656f6f1412221, State: Initialized, Role: FOLLOWER
I20251212 08:52:39.223855 1790 consensus_queue.cc:260] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } }
I20251212 08:52:39.224014 1790 raft_consensus.cc:399] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:39.224133 1790 raft_consensus.cc:493] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:39.224256 1790 raft_consensus.cc:3060] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:39.226328 1790 raft_consensus.cc:515] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } }
I20251212 08:52:39.226516 1790 leader_election.cc:304] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 511fc2eb89c64ea7819656f6f1412221; no voters:
I20251212 08:52:39.226715 1790 leader_election.cc:290] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:39.226778 1792 raft_consensus.cc:2804] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:39.226958 1792 raft_consensus.cc:697] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 1 LEADER]: Becoming Leader. State: Replica: 511fc2eb89c64ea7819656f6f1412221, State: Running, Role: LEADER
I20251212 08:52:39.227133 1790 ts_tablet_manager.cc:1434] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20251212 08:52:39.227200 1792 consensus_queue.cc:237] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } }
I20251212 08:52:39.229336 1635 catalog_manager.cc:5654] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 reported cstate change: term changed from 0 to 1, leader changed from <none> to 511fc2eb89c64ea7819656f6f1412221 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "511fc2eb89c64ea7819656f6f1412221" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "511fc2eb89c64ea7819656f6f1412221" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46651 } health_report { overall_health: HEALTHY } } }
W20251212 08:52:39.242249 1785 meta_cache.cc:788] Not found: LookupRpcById { tablet: 'dummy-tablet-id', attempt: 1 } failed
I20251212 08:52:39.243135 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:52:39.251850 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:52:39.252318 832 tablet_replica.cc:333] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221: stopping tablet replica
I20251212 08:52:39.252516 832 raft_consensus.cc:2243] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:39.252729 832 raft_consensus.cc:2272] T 81e281d7e3564cada927454855df539f P 511fc2eb89c64ea7819656f6f1412221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:39.253650 832 tablet_replica.cc:333] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221: stopping tablet replica
I20251212 08:52:39.253813 832 raft_consensus.cc:2243] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:39.254047 832 raft_consensus.cc:2272] T 6ccdbdd5ecdb443eaef05f9b59595dae P 511fc2eb89c64ea7819656f6f1412221 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:39.268927 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:52:39.273628 832 master.cc:561] Master@127.0.208.62:46479 shutting down...
I20251212 08:52:39.282254 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:39.282528 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:39.282730 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P b898e502592d488cb990963c000cc14d: stopping tablet replica
I20251212 08:52:39.296949 832 master.cc:583] Master@127.0.208.62:46479 shutdown complete.
[ OK ] ClientTest.TestBasicIdBasedLookup (283 ms)
[ RUN ] ClientTest.TestScannerKeepAlive
I20251212 08:52:39.306836 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:38827
I20251212 08:52:39.307549 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:39.309968 1799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:39.310346 1800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:39.310637 1802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:39.310889 832 server_base.cc:1047] running on GCE node
I20251212 08:52:39.311156 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:39.311235 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:39.311281 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529559311267 us; error 0 us; skew 500 ppm
I20251212 08:52:39.311424 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:39.312523 832 webserver.cc:492] Webserver started at http://127.0.208.62:46041/ using document root <none> and password file <none>
I20251212 08:52:39.312736 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:39.312805 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:39.312920 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:39.313506 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "46a6966e66494991bb947e3429b98b6a"
format_stamp: "Formatted at 2025-12-12 08:52:39 on dist-test-slave-qdpb"
I20251212 08:52:39.315897 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:52:39.317481 1807 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:39.317799 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:52:39.317915 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "46a6966e66494991bb947e3429b98b6a"
format_stamp: "Formatted at 2025-12-12 08:52:39 on dist-test-slave-qdpb"
I20251212 08:52:39.318039 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:39.349272 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:39.349845 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:39.364743 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:38827
I20251212 08:52:39.364775 1868 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:38827 every 8 connection(s)
I20251212 08:52:39.366036 1869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:39.370244 1869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a: Bootstrap starting.
I20251212 08:52:39.371598 1869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:39.373169 1869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a: No bootstrap required, opened a new log
I20251212 08:52:39.373811 1869 raft_consensus.cc:359] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a6966e66494991bb947e3429b98b6a" member_type: VOTER }
I20251212 08:52:39.373946 1869 raft_consensus.cc:385] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:39.374050 1869 raft_consensus.cc:740] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46a6966e66494991bb947e3429b98b6a, State: Initialized, Role: FOLLOWER
I20251212 08:52:39.374261 1869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a6966e66494991bb947e3429b98b6a" member_type: VOTER }
I20251212 08:52:39.374399 1869 raft_consensus.cc:399] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:39.374480 1869 raft_consensus.cc:493] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:39.374591 1869 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:39.376403 1869 raft_consensus.cc:515] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a6966e66494991bb947e3429b98b6a" member_type: VOTER }
I20251212 08:52:39.376617 1869 leader_election.cc:304] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46a6966e66494991bb947e3429b98b6a; no voters:
I20251212 08:52:39.376984 1869 leader_election.cc:290] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:39.377100 1872 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:39.377554 1872 raft_consensus.cc:697] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 1 LEADER]: Becoming Leader. State: Replica: 46a6966e66494991bb947e3429b98b6a, State: Running, Role: LEADER
I20251212 08:52:39.377822 1872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a6966e66494991bb947e3429b98b6a" member_type: VOTER }
I20251212 08:52:39.377871 1869 sys_catalog.cc:565] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:39.378729 1873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46a6966e66494991bb947e3429b98b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a6966e66494991bb947e3429b98b6a" member_type: VOTER } }
I20251212 08:52:39.378923 1873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:39.378935 1874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46a6966e66494991bb947e3429b98b6a. Latest consensus state: current_term: 1 leader_uuid: "46a6966e66494991bb947e3429b98b6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46a6966e66494991bb947e3429b98b6a" member_type: VOTER } }
I20251212 08:52:39.379180 1874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:39.380417 1877 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:39.381736 1877 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:39.384696 1877 catalog_manager.cc:1357] Generated new cluster ID: 13f998a44e9f4ef690704fd6eac4ea07
I20251212 08:52:39.384817 1877 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:39.385856 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:39.399744 1877 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:39.400645 1877 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:39.412647 1877 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a: Generated new TSK 0
I20251212 08:52:39.412928 1877 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:39.418572 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:39.421746 1891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:39.421828 1890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:39.422545 1893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:39.423223 832 server_base.cc:1047] running on GCE node
I20251212 08:52:39.423489 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:39.423552 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:39.423606 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529559423574 us; error 0 us; skew 500 ppm
I20251212 08:52:39.423758 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:39.424930 832 webserver.cc:492] Webserver started at http://127.0.208.1:40511/ using document root <none> and password file <none>
I20251212 08:52:39.425143 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:39.425244 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:39.425395 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:39.426049 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "2a4cfe8642b642dfb0f73d70369812d3"
format_stamp: "Formatted at 2025-12-12 08:52:39 on dist-test-slave-qdpb"
I20251212 08:52:39.428660 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:52:39.430262 1898 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:39.430661 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:52:39.430760 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "2a4cfe8642b642dfb0f73d70369812d3"
format_stamp: "Formatted at 2025-12-12 08:52:39 on dist-test-slave-qdpb"
I20251212 08:52:39.430874 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestScannerKeepAlive.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:39.452936 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:39.453550 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:39.455147 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:39.455271 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:39.455395 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:39.455506 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:39.483714 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35771
I20251212 08:52:39.483750 1968 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35771 every 8 connection(s)
I20251212 08:52:39.485598 1969 heartbeater.cc:344] Connected to a master server at 127.0.208.62:38827
I20251212 08:52:39.485728 1969 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:39.486166 1969 heartbeater.cc:507] Master 127.0.208.62:38827 requested a full tablet report, sending...
I20251212 08:52:39.486971 1824 ts_manager.cc:194] Registered new tserver with Master: 2a4cfe8642b642dfb0f73d70369812d3 (127.0.208.1:35771)
I20251212 08:52:39.487016 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002402981s
I20251212 08:52:39.488178 1824 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38278
I20251212 08:52:39.499670 1969 heartbeater.cc:499] Master 127.0.208.62:38827 was elected leader, sending a full tablet report...
I20251212 08:52:39.502025 1824 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38286:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:39.518487 1929 tablet_service.cc:1505] Processing CreateTablet for tablet 25cbc83bab9d402eb9cba481a000074e (DEFAULT_TABLE table=client-testtb [id=d584d7d4cedc458d82c01fc592fdb947]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:39.518568 1928 tablet_service.cc:1505] Processing CreateTablet for tablet ab7040d6ca3a43f39d1d6abe7b54ec3d (DEFAULT_TABLE table=client-testtb [id=d584d7d4cedc458d82c01fc592fdb947]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:39.519078 1929 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25cbc83bab9d402eb9cba481a000074e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:39.519304 1928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab7040d6ca3a43f39d1d6abe7b54ec3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:39.525611 1979 tablet_bootstrap.cc:492] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3: Bootstrap starting.
I20251212 08:52:39.527038 1979 tablet_bootstrap.cc:654] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:39.528870 1979 tablet_bootstrap.cc:492] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3: No bootstrap required, opened a new log
I20251212 08:52:39.528999 1979 ts_tablet_manager.cc:1403] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251212 08:52:39.529590 1979 raft_consensus.cc:359] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } }
I20251212 08:52:39.529731 1979 raft_consensus.cc:385] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:39.529798 1979 raft_consensus.cc:740] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4cfe8642b642dfb0f73d70369812d3, State: Initialized, Role: FOLLOWER
I20251212 08:52:39.530449 1979 consensus_queue.cc:260] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } }
I20251212 08:52:39.530583 1979 raft_consensus.cc:399] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:39.530637 1979 raft_consensus.cc:493] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:39.530722 1979 raft_consensus.cc:3060] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:39.532496 1979 raft_consensus.cc:515] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } }
I20251212 08:52:39.532687 1979 leader_election.cc:304] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a4cfe8642b642dfb0f73d70369812d3; no voters:
I20251212 08:52:39.533089 1979 leader_election.cc:290] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:39.533273 1981 raft_consensus.cc:2804] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:39.533921 1979 ts_tablet_manager.cc:1434] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:52:39.533915 1981 raft_consensus.cc:697] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 LEADER]: Becoming Leader. State: Replica: 2a4cfe8642b642dfb0f73d70369812d3, State: Running, Role: LEADER
I20251212 08:52:39.534514 1979 tablet_bootstrap.cc:492] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3: Bootstrap starting.
I20251212 08:52:39.534626 1981 consensus_queue.cc:237] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } }
I20251212 08:52:39.536224 1979 tablet_bootstrap.cc:654] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:39.536926 1823 catalog_manager.cc:5654] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a4cfe8642b642dfb0f73d70369812d3 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "2a4cfe8642b642dfb0f73d70369812d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:39.538136 1979 tablet_bootstrap.cc:492] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3: No bootstrap required, opened a new log
I20251212 08:52:39.538275 1979 ts_tablet_manager.cc:1403] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:39.538964 1979 raft_consensus.cc:359] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } }
I20251212 08:52:39.539141 1979 raft_consensus.cc:385] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:39.539215 1979 raft_consensus.cc:740] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a4cfe8642b642dfb0f73d70369812d3, State: Initialized, Role: FOLLOWER
I20251212 08:52:39.539500 1979 consensus_queue.cc:260] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } }
I20251212 08:52:39.539633 1979 raft_consensus.cc:399] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:39.539745 1979 raft_consensus.cc:493] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:39.540398 1979 raft_consensus.cc:3060] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:39.542661 1979 raft_consensus.cc:515] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } }
I20251212 08:52:39.542847 1979 leader_election.cc:304] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2a4cfe8642b642dfb0f73d70369812d3; no voters:
I20251212 08:52:39.543045 1979 leader_election.cc:290] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:39.543088 1981 raft_consensus.cc:2804] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:39.543273 1981 raft_consensus.cc:697] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 LEADER]: Becoming Leader. State: Replica: 2a4cfe8642b642dfb0f73d70369812d3, State: Running, Role: LEADER
I20251212 08:52:39.543476 1979 ts_tablet_manager.cc:1434] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20251212 08:52:39.543501 1981 consensus_queue.cc:237] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } }
I20251212 08:52:39.545209 1823 catalog_manager.cc:5654] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2a4cfe8642b642dfb0f73d70369812d3 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "2a4cfe8642b642dfb0f73d70369812d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2a4cfe8642b642dfb0f73d70369812d3" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35771 } health_report { overall_health: HEALTHY } } }
W20251212 08:52:41.201141 1975 client.cc:2042] Couldn't close scanner 4b25f8053f9843b6b2a36dfd8f072037: Service unavailable: reactor is shutting down (error 108)
I20251212 08:52:41.201131 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
W20251212 08:52:41.201536 1902 connection.cc:537] server connection from 127.0.0.1:45302 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251212 08:52:41.210573 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:52:41.210930 832 tablet_replica.cc:333] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3: stopping tablet replica
I20251212 08:52:41.211138 832 raft_consensus.cc:2243] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:41.211364 832 raft_consensus.cc:2272] T 25cbc83bab9d402eb9cba481a000074e P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:41.212044 832 tablet_replica.cc:333] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3: stopping tablet replica
I20251212 08:52:41.212189 832 raft_consensus.cc:2243] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:41.212391 832 raft_consensus.cc:2272] T ab7040d6ca3a43f39d1d6abe7b54ec3d P 2a4cfe8642b642dfb0f73d70369812d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:41.226399 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:52:41.230394 832 master.cc:561] Master@127.0.208.62:38827 shutting down...
I20251212 08:52:41.240782 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:41.241062 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:41.241164 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 46a6966e66494991bb947e3429b98b6a: stopping tablet replica
I20251212 08:52:41.254462 832 master.cc:583] Master@127.0.208.62:38827 shutdown complete.
[ OK ] ClientTest.TestScannerKeepAlive (1957 ms)
[ RUN ] ClientTest.TestInsertAutoFlushSync
I20251212 08:52:41.264638 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:46521
I20251212 08:52:41.265168 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:41.267679 1991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:41.267680 1992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:41.268345 1994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:41.268639 832 server_base.cc:1047] running on GCE node
I20251212 08:52:41.268921 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:41.268985 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:41.269080 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529561269027 us; error 0 us; skew 500 ppm
I20251212 08:52:41.269248 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:41.270359 832 webserver.cc:492] Webserver started at http://127.0.208.62:39175/ using document root <none> and password file <none>
I20251212 08:52:41.270568 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:41.270653 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:41.270783 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:41.271411 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "bc51263c4cd34beabf0c067d2b9fbeff"
format_stamp: "Formatted at 2025-12-12 08:52:41 on dist-test-slave-qdpb"
I20251212 08:52:41.273803 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20251212 08:52:41.275318 1999 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:41.275640 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:52:41.275748 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "bc51263c4cd34beabf0c067d2b9fbeff"
format_stamp: "Formatted at 2025-12-12 08:52:41 on dist-test-slave-qdpb"
I20251212 08:52:41.275882 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:41.288661 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:41.289278 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:41.304751 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:46521
I20251212 08:52:41.304795 2060 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:46521 every 8 connection(s)
I20251212 08:52:41.306142 2061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:41.310307 2061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff: Bootstrap starting.
I20251212 08:52:41.311659 2061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:41.313299 2061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff: No bootstrap required, opened a new log
I20251212 08:52:41.314010 2061 raft_consensus.cc:359] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51263c4cd34beabf0c067d2b9fbeff" member_type: VOTER }
I20251212 08:52:41.314157 2061 raft_consensus.cc:385] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:41.314374 2061 raft_consensus.cc:740] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc51263c4cd34beabf0c067d2b9fbeff, State: Initialized, Role: FOLLOWER
I20251212 08:52:41.314620 2061 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51263c4cd34beabf0c067d2b9fbeff" member_type: VOTER }
I20251212 08:52:41.314762 2061 raft_consensus.cc:399] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:41.314855 2061 raft_consensus.cc:493] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:41.314958 2061 raft_consensus.cc:3060] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:41.316849 2061 raft_consensus.cc:515] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51263c4cd34beabf0c067d2b9fbeff" member_type: VOTER }
I20251212 08:52:41.317063 2061 leader_election.cc:304] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc51263c4cd34beabf0c067d2b9fbeff; no voters:
I20251212 08:52:41.317466 2061 leader_election.cc:290] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:41.317643 2064 raft_consensus.cc:2804] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:41.318192 2064 raft_consensus.cc:697] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 1 LEADER]: Becoming Leader. State: Replica: bc51263c4cd34beabf0c067d2b9fbeff, State: Running, Role: LEADER
I20251212 08:52:41.318364 2061 sys_catalog.cc:565] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:41.318462 2064 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51263c4cd34beabf0c067d2b9fbeff" member_type: VOTER }
I20251212 08:52:41.319461 2066 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc51263c4cd34beabf0c067d2b9fbeff. Latest consensus state: current_term: 1 leader_uuid: "bc51263c4cd34beabf0c067d2b9fbeff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51263c4cd34beabf0c067d2b9fbeff" member_type: VOTER } }
I20251212 08:52:41.319718 2066 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:41.319438 2065 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc51263c4cd34beabf0c067d2b9fbeff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51263c4cd34beabf0c067d2b9fbeff" member_type: VOTER } }
I20251212 08:52:41.320662 2065 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:41.320864 2071 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:41.322619 2071 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:41.323701 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:41.325557 2071 catalog_manager.cc:1357] Generated new cluster ID: 2242c4df328d4208af8e3d4e026b20b8
I20251212 08:52:41.325631 2071 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:41.344035 2071 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:41.344798 2071 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:41.359820 2071 catalog_manager.cc:6027] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff: Generated new TSK 0
I20251212 08:52:41.360042 2071 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:41.388913 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:41.392362 2082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:41.392379 2083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:41.393107 2085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:41.393801 832 server_base.cc:1047] running on GCE node
I20251212 08:52:41.394106 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:41.394193 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:41.394253 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529561394234 us; error 0 us; skew 500 ppm
I20251212 08:52:41.394418 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:41.395565 832 webserver.cc:492] Webserver started at http://127.0.208.1:46733/ using document root <none> and password file <none>
I20251212 08:52:41.395812 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:41.395895 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:41.396052 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:41.396865 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "ad8e97a15d3742c4863b597b7bbf612b"
format_stamp: "Formatted at 2025-12-12 08:52:41 on dist-test-slave-qdpb"
I20251212 08:52:41.399467 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:52:41.401146 2090 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:41.401614 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:52:41.401774 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "ad8e97a15d3742c4863b597b7bbf612b"
format_stamp: "Formatted at 2025-12-12 08:52:41 on dist-test-slave-qdpb"
I20251212 08:52:41.401901 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertAutoFlushSync.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:41.411211 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:41.411813 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:41.414198 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:41.414314 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:41.414448 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:41.414513 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:41.439812 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:34653
I20251212 08:52:41.439833 2160 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:34653 every 8 connection(s)
I20251212 08:52:41.441630 2161 heartbeater.cc:344] Connected to a master server at 127.0.208.62:46521
I20251212 08:52:41.441787 2161 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:41.442195 2161 heartbeater.cc:507] Master 127.0.208.62:46521 requested a full tablet report, sending...
I20251212 08:52:41.443082 2016 ts_manager.cc:194] Registered new tserver with Master: ad8e97a15d3742c4863b597b7bbf612b (127.0.208.1:34653)
I20251212 08:52:41.443090 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002386146s
I20251212 08:52:41.444331 2016 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43986
I20251212 08:52:41.456502 2161 heartbeater.cc:499] Master 127.0.208.62:46521 was elected leader, sending a full tablet report...
I20251212 08:52:41.456404 2016 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44014:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:41.478070 2121 tablet_service.cc:1505] Processing CreateTablet for tablet fd2753c4c11c42e69297d52311426201 (DEFAULT_TABLE table=client-testtb [id=a2f13c7f0a2a45539c9f08010d1655ae]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:41.478399 2120 tablet_service.cc:1505] Processing CreateTablet for tablet a4b8ffca49bd4537a3057b8971c972f7 (DEFAULT_TABLE table=client-testtb [id=a2f13c7f0a2a45539c9f08010d1655ae]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:41.479130 2121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd2753c4c11c42e69297d52311426201. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:41.479605 2120 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a4b8ffca49bd4537a3057b8971c972f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:41.487643 2171 tablet_bootstrap.cc:492] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b: Bootstrap starting.
I20251212 08:52:41.489445 2171 tablet_bootstrap.cc:654] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:41.491703 2171 tablet_bootstrap.cc:492] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b: No bootstrap required, opened a new log
I20251212 08:52:41.491838 2171 ts_tablet_manager.cc:1403] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:41.492626 2171 raft_consensus.cc:359] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } }
I20251212 08:52:41.492801 2171 raft_consensus.cc:385] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:41.492903 2171 raft_consensus.cc:740] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8e97a15d3742c4863b597b7bbf612b, State: Initialized, Role: FOLLOWER
I20251212 08:52:41.493155 2171 consensus_queue.cc:260] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } }
I20251212 08:52:41.493319 2171 raft_consensus.cc:399] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:41.493404 2171 raft_consensus.cc:493] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:41.493536 2171 raft_consensus.cc:3060] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:41.496275 2171 raft_consensus.cc:515] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } }
I20251212 08:52:41.496697 2171 leader_election.cc:304] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad8e97a15d3742c4863b597b7bbf612b; no voters:
I20251212 08:52:41.497287 2171 leader_election.cc:290] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:41.497485 2173 raft_consensus.cc:2804] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:41.498461 2171 ts_tablet_manager.cc:1434] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20251212 08:52:41.498834 2171 tablet_bootstrap.cc:492] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b: Bootstrap starting.
I20251212 08:52:41.499130 2173 raft_consensus.cc:697] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 1 LEADER]: Becoming Leader. State: Replica: ad8e97a15d3742c4863b597b7bbf612b, State: Running, Role: LEADER
I20251212 08:52:41.499639 2173 consensus_queue.cc:237] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } }
I20251212 08:52:41.500932 2171 tablet_bootstrap.cc:654] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:41.503528 2015 catalog_manager.cc:5654] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b reported cstate change: term changed from 0 to 1, leader changed from <none> to ad8e97a15d3742c4863b597b7bbf612b (127.0.208.1). New cstate: current_term: 1 leader_uuid: "ad8e97a15d3742c4863b597b7bbf612b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:41.505216 2171 tablet_bootstrap.cc:492] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b: No bootstrap required, opened a new log
I20251212 08:52:41.505371 2171 ts_tablet_manager.cc:1403] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20251212 08:52:41.506057 2171 raft_consensus.cc:359] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } }
I20251212 08:52:41.506245 2171 raft_consensus.cc:385] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:41.506340 2171 raft_consensus.cc:740] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8e97a15d3742c4863b597b7bbf612b, State: Initialized, Role: FOLLOWER
I20251212 08:52:41.506615 2171 consensus_queue.cc:260] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } }
I20251212 08:52:41.506748 2171 raft_consensus.cc:399] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:41.506839 2171 raft_consensus.cc:493] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:41.506964 2171 raft_consensus.cc:3060] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:41.509202 2171 raft_consensus.cc:515] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } }
I20251212 08:52:41.509382 2171 leader_election.cc:304] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad8e97a15d3742c4863b597b7bbf612b; no voters:
I20251212 08:52:41.509577 2171 leader_election.cc:290] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:41.509634 2173 raft_consensus.cc:2804] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:41.509824 2173 raft_consensus.cc:697] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 1 LEADER]: Becoming Leader. State: Replica: ad8e97a15d3742c4863b597b7bbf612b, State: Running, Role: LEADER
I20251212 08:52:41.510027 2171 ts_tablet_manager.cc:1434] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b: Time spent starting tablet: real 0.005s user 0.000s sys 0.004s
I20251212 08:52:41.510071 2173 consensus_queue.cc:237] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } }
I20251212 08:52:41.511960 2016 catalog_manager.cc:5654] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b reported cstate change: term changed from 0 to 1, leader changed from <none> to ad8e97a15d3742c4863b597b7bbf612b (127.0.208.1). New cstate: current_term: 1 leader_uuid: "ad8e97a15d3742c4863b597b7bbf612b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad8e97a15d3742c4863b597b7bbf612b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34653 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:41.826274 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:52:41.834626 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:52:41.835045 832 tablet_replica.cc:333] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b: stopping tablet replica
I20251212 08:52:41.835253 832 raft_consensus.cc:2243] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:41.835457 832 raft_consensus.cc:2272] T a4b8ffca49bd4537a3057b8971c972f7 P ad8e97a15d3742c4863b597b7bbf612b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:41.836473 832 tablet_replica.cc:333] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b: stopping tablet replica
I20251212 08:52:41.836627 832 raft_consensus.cc:2243] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:41.836824 832 raft_consensus.cc:2272] T fd2753c4c11c42e69297d52311426201 P ad8e97a15d3742c4863b597b7bbf612b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:41.842350 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:52:41.846717 832 master.cc:561] Master@127.0.208.62:46521 shutting down...
I20251212 08:52:41.855701 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:41.855952 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:41.856105 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P bc51263c4cd34beabf0c067d2b9fbeff: stopping tablet replica
I20251212 08:52:41.859901 832 master.cc:583] Master@127.0.208.62:46521 shutdown complete.
[ OK ] ClientTest.TestInsertAutoFlushSync (604 ms)
[ RUN ] ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn
I20251212 08:52:41.869493 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:38075
I20251212 08:52:41.870046 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:41.872659 2182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:41.872762 2183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:41.872990 2185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:41.873279 832 server_base.cc:1047] running on GCE node
I20251212 08:52:41.873575 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:41.873653 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:41.873734 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529561873698 us; error 0 us; skew 500 ppm
I20251212 08:52:41.873891 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:41.875047 832 webserver.cc:492] Webserver started at http://127.0.208.62:35503/ using document root <none> and password file <none>
I20251212 08:52:41.875275 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:41.875377 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:41.875505 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:41.876154 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "0d8675787e5342cfaccb8a8f54db89c6"
format_stamp: "Formatted at 2025-12-12 08:52:41 on dist-test-slave-qdpb"
I20251212 08:52:41.878608 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:41.880223 2190 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:41.880661 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:52:41.880764 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "0d8675787e5342cfaccb8a8f54db89c6"
format_stamp: "Formatted at 2025-12-12 08:52:41 on dist-test-slave-qdpb"
I20251212 08:52:41.880877 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:41.889266 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:41.889829 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:41.904714 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:38075
I20251212 08:52:41.904776 2251 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:38075 every 8 connection(s)
I20251212 08:52:41.906030 2252 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:41.910243 2252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6: Bootstrap starting.
I20251212 08:52:41.911640 2252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:41.913321 2252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6: No bootstrap required, opened a new log
I20251212 08:52:41.913995 2252 raft_consensus.cc:359] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8675787e5342cfaccb8a8f54db89c6" member_type: VOTER }
I20251212 08:52:41.914131 2252 raft_consensus.cc:385] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:41.914225 2252 raft_consensus.cc:740] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d8675787e5342cfaccb8a8f54db89c6, State: Initialized, Role: FOLLOWER
I20251212 08:52:41.914454 2252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8675787e5342cfaccb8a8f54db89c6" member_type: VOTER }
I20251212 08:52:41.914589 2252 raft_consensus.cc:399] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:41.914666 2252 raft_consensus.cc:493] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:41.914777 2252 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:41.916692 2252 raft_consensus.cc:515] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8675787e5342cfaccb8a8f54db89c6" member_type: VOTER }
I20251212 08:52:41.916913 2252 leader_election.cc:304] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d8675787e5342cfaccb8a8f54db89c6; no voters:
I20251212 08:52:41.917331 2252 leader_election.cc:290] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:41.917511 2255 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:41.918011 2255 raft_consensus.cc:697] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 1 LEADER]: Becoming Leader. State: Replica: 0d8675787e5342cfaccb8a8f54db89c6, State: Running, Role: LEADER
I20251212 08:52:41.918255 2252 sys_catalog.cc:565] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:41.918298 2255 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8675787e5342cfaccb8a8f54db89c6" member_type: VOTER }
I20251212 08:52:41.919201 2256 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d8675787e5342cfaccb8a8f54db89c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8675787e5342cfaccb8a8f54db89c6" member_type: VOTER } }
I20251212 08:52:41.919225 2257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d8675787e5342cfaccb8a8f54db89c6. Latest consensus state: current_term: 1 leader_uuid: "0d8675787e5342cfaccb8a8f54db89c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d8675787e5342cfaccb8a8f54db89c6" member_type: VOTER } }
I20251212 08:52:41.919400 2257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:41.919404 2256 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:41.920756 2261 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:41.922282 2261 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:41.923400 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:41.925149 2261 catalog_manager.cc:1357] Generated new cluster ID: 69c3d4f06c9a401888e4e328ab616132
I20251212 08:52:41.925239 2261 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:41.941535 2261 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:41.942361 2261 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:41.952745 2261 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6: Generated new TSK 0
I20251212 08:52:41.953020 2261 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:41.956198 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:41.959404 2274 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:41.959812 2273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:41.960178 832 server_base.cc:1047] running on GCE node
W20251212 08:52:41.960215 2276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:41.960590 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:41.960664 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:41.960702 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529561960687 us; error 0 us; skew 500 ppm
I20251212 08:52:41.960834 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:41.962183 832 webserver.cc:492] Webserver started at http://127.0.208.1:35835/ using document root <none> and password file <none>
I20251212 08:52:41.962384 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:41.962455 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:41.962570 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:41.963227 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "9cc9c51370b44cff841a7b56edd2bd8a"
format_stamp: "Formatted at 2025-12-12 08:52:41 on dist-test-slave-qdpb"
I20251212 08:52:41.965858 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:52:41.967576 2281 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:41.968062 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:52:41.968214 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "9cc9c51370b44cff841a7b56edd2bd8a"
format_stamp: "Formatted at 2025-12-12 08:52:41 on dist-test-slave-qdpb"
I20251212 08:52:41.968401 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:41.990253 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:41.990833 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:41.992551 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:41.992679 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:41.992825 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:41.992892 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.017793 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:39327
I20251212 08:52:42.017815 2351 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:39327 every 8 connection(s)
I20251212 08:52:42.019613 2352 heartbeater.cc:344] Connected to a master server at 127.0.208.62:38075
I20251212 08:52:42.019755 2352 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:42.020720 2352 heartbeater.cc:507] Master 127.0.208.62:38075 requested a full tablet report, sending...
I20251212 08:52:42.021613 2207 ts_manager.cc:194] Registered new tserver with Master: 9cc9c51370b44cff841a7b56edd2bd8a (127.0.208.1:39327)
I20251212 08:52:42.022300 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.003496577s
I20251212 08:52:42.022817 2207 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59668
I20251212 08:52:42.033499 2207 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59682:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:42.034003 2352 heartbeater.cc:499] Master 127.0.208.62:38075 was elected leader, sending a full tablet report...
I20251212 08:52:42.048636 2311 tablet_service.cc:1505] Processing CreateTablet for tablet 66549ab4d2764cadb22aa495ff156505 (DEFAULT_TABLE table=client-testtb [id=11d4ce30093440a586352b0b0c253b05]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:42.048602 2312 tablet_service.cc:1505] Processing CreateTablet for tablet d26bfee1f1074a4bbbbf4443bbcbd73d (DEFAULT_TABLE table=client-testtb [id=11d4ce30093440a586352b0b0c253b05]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:42.049327 2312 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d26bfee1f1074a4bbbbf4443bbcbd73d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.049577 2311 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66549ab4d2764cadb22aa495ff156505. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.058092 2362 tablet_bootstrap.cc:492] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a: Bootstrap starting.
I20251212 08:52:42.059988 2362 tablet_bootstrap.cc:654] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.061896 2362 tablet_bootstrap.cc:492] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a: No bootstrap required, opened a new log
I20251212 08:52:42.062068 2362 ts_tablet_manager.cc:1403] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:42.062736 2362 raft_consensus.cc:359] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } }
I20251212 08:52:42.062891 2362 raft_consensus.cc:385] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.062994 2362 raft_consensus.cc:740] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cc9c51370b44cff841a7b56edd2bd8a, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.063215 2362 consensus_queue.cc:260] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } }
I20251212 08:52:42.063365 2362 raft_consensus.cc:399] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:42.063442 2362 raft_consensus.cc:493] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:42.063567 2362 raft_consensus.cc:3060] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.065711 2362 raft_consensus.cc:515] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } }
I20251212 08:52:42.065973 2362 leader_election.cc:304] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cc9c51370b44cff841a7b56edd2bd8a; no voters:
I20251212 08:52:42.066429 2362 leader_election.cc:290] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:42.066650 2364 raft_consensus.cc:2804] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:42.067184 2362 ts_tablet_manager.cc:1434] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20251212 08:52:42.067488 2362 tablet_bootstrap.cc:492] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a: Bootstrap starting.
I20251212 08:52:42.067592 2364 raft_consensus.cc:697] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 LEADER]: Becoming Leader. State: Replica: 9cc9c51370b44cff841a7b56edd2bd8a, State: Running, Role: LEADER
I20251212 08:52:42.068482 2364 consensus_queue.cc:237] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } }
I20251212 08:52:42.069540 2362 tablet_bootstrap.cc:654] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.071206 2206 catalog_manager.cc:5654] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cc9c51370b44cff841a7b56edd2bd8a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:42.071877 2362 tablet_bootstrap.cc:492] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a: No bootstrap required, opened a new log
I20251212 08:52:42.072046 2362 ts_tablet_manager.cc:1403] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251212 08:52:42.072840 2362 raft_consensus.cc:359] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } }
I20251212 08:52:42.073063 2362 raft_consensus.cc:385] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.073143 2362 raft_consensus.cc:740] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cc9c51370b44cff841a7b56edd2bd8a, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.073388 2362 consensus_queue.cc:260] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } }
I20251212 08:52:42.073555 2362 raft_consensus.cc:399] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:42.073660 2362 raft_consensus.cc:493] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:42.073774 2362 raft_consensus.cc:3060] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.075981 2362 raft_consensus.cc:515] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } }
I20251212 08:52:42.076181 2362 leader_election.cc:304] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cc9c51370b44cff841a7b56edd2bd8a; no voters:
I20251212 08:52:42.076442 2362 leader_election.cc:290] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:42.076494 2364 raft_consensus.cc:2804] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:42.076659 2364 raft_consensus.cc:697] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 LEADER]: Becoming Leader. State: Replica: 9cc9c51370b44cff841a7b56edd2bd8a, State: Running, Role: LEADER
I20251212 08:52:42.076934 2362 ts_tablet_manager.cc:1434] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20251212 08:52:42.076882 2364 consensus_queue.cc:237] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } }
I20251212 08:52:42.078882 2206 catalog_manager.cc:5654] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 9cc9c51370b44cff841a7b56edd2bd8a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cc9c51370b44cff841a7b56edd2bd8a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 39327 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:42.086005 2206 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59682:
table { table_name: "client-testtb" } alter_schema_steps { type: ALTER_COLUMN alter_column { delta { name: "non_null_with_default" remove_default: true } } } modify_external_catalogs: true
I20251212 08:52:42.092084 2372 tablet.cc:1722] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251212 08:52:42.092633 2373 tablet.cc:1722] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251212 08:52:42.093693 2373 tablet.cc:1620] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a: MemRowSet was empty: no flush needed.
I20251212 08:52:42.093698 2372 tablet.cc:1620] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a: MemRowSet was empty: no flush needed.
I20251212 08:52:42.102083 2206 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6: client-testtb [id=11d4ce30093440a586352b0b0c253b05] alter complete (version 1)
I20251212 08:52:42.122485 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:52:42.131209 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:52:42.131654 832 tablet_replica.cc:333] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a: stopping tablet replica
I20251212 08:52:42.131850 832 raft_consensus.cc:2243] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:42.132094 832 raft_consensus.cc:2272] T d26bfee1f1074a4bbbbf4443bbcbd73d P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:42.132975 832 tablet_replica.cc:333] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a: stopping tablet replica
I20251212 08:52:42.133134 832 raft_consensus.cc:2243] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:42.133306 832 raft_consensus.cc:2272] T 66549ab4d2764cadb22aa495ff156505 P 9cc9c51370b44cff841a7b56edd2bd8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:42.138259 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:52:42.143314 832 master.cc:561] Master@127.0.208.62:38075 shutting down...
I20251212 08:52:42.165059 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:42.165326 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:42.165468 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 0d8675787e5342cfaccb8a8f54db89c6: stopping tablet replica
I20251212 08:52:42.180222 832 master.cc:583] Master@127.0.208.62:38075 shutdown complete.
[ OK ] ClientTest.TestBatchWithPartialErrorOfMissingRequiredColumn (321 ms)
[ RUN ] ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered
I20251212 08:52:42.190536 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:36405
I20251212 08:52:42.191066 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:42.193513 2375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.193641 2376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.194048 2378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:42.195094 832 server_base.cc:1047] running on GCE node
I20251212 08:52:42.195421 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:42.195477 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:42.195519 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529562195501 us; error 0 us; skew 500 ppm
I20251212 08:52:42.195670 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:42.197227 832 webserver.cc:492] Webserver started at http://127.0.208.62:44839/ using document root <none> and password file <none>
I20251212 08:52:42.197438 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:42.197502 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:42.197611 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:42.198244 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "9265c238c8a944b794fbfb1d6854b5f6"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.201252 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20251212 08:52:42.203114 2383 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.203549 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:42.203755 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "9265c238c8a944b794fbfb1d6854b5f6"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.203892 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:42.223376 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:42.223927 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:42.239454 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:36405
I20251212 08:52:42.239493 2444 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:36405 every 8 connection(s)
I20251212 08:52:42.240777 2445 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.244853 2445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6: Bootstrap starting.
I20251212 08:52:42.246115 2445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.247774 2445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6: No bootstrap required, opened a new log
I20251212 08:52:42.248535 2445 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9265c238c8a944b794fbfb1d6854b5f6" member_type: VOTER }
I20251212 08:52:42.248670 2445 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.248713 2445 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9265c238c8a944b794fbfb1d6854b5f6, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.248914 2445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9265c238c8a944b794fbfb1d6854b5f6" member_type: VOTER }
I20251212 08:52:42.249032 2445 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:42.249081 2445 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:42.249141 2445 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.250959 2445 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9265c238c8a944b794fbfb1d6854b5f6" member_type: VOTER }
I20251212 08:52:42.251137 2445 leader_election.cc:304] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9265c238c8a944b794fbfb1d6854b5f6; no voters:
I20251212 08:52:42.251564 2445 leader_election.cc:290] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:42.251724 2448 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:42.252175 2448 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 1 LEADER]: Becoming Leader. State: Replica: 9265c238c8a944b794fbfb1d6854b5f6, State: Running, Role: LEADER
I20251212 08:52:42.252467 2445 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:42.252432 2448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9265c238c8a944b794fbfb1d6854b5f6" member_type: VOTER }
I20251212 08:52:42.253288 2450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9265c238c8a944b794fbfb1d6854b5f6. Latest consensus state: current_term: 1 leader_uuid: "9265c238c8a944b794fbfb1d6854b5f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9265c238c8a944b794fbfb1d6854b5f6" member_type: VOTER } }
I20251212 08:52:42.253468 2450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:42.253288 2449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9265c238c8a944b794fbfb1d6854b5f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9265c238c8a944b794fbfb1d6854b5f6" member_type: VOTER } }
I20251212 08:52:42.253592 2449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:42.254961 2453 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:42.257086 2453 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:42.257892 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:42.260313 2453 catalog_manager.cc:1357] Generated new cluster ID: 1d30d1677a094d959e57a95af6bdab44
I20251212 08:52:42.260401 2453 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:42.269340 2453 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:42.270135 2453 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:42.279850 2453 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6: Generated new TSK 0
I20251212 08:52:42.280136 2453 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:42.291045 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:42.294874 2466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:42.295958 832 server_base.cc:1047] running on GCE node
W20251212 08:52:42.295976 2467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.296312 2469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:42.296725 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:42.296808 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:42.296847 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529562296831 us; error 0 us; skew 500 ppm
I20251212 08:52:42.297046 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:42.298326 832 webserver.cc:492] Webserver started at http://127.0.208.1:45775/ using document root <none> and password file <none>
I20251212 08:52:42.298552 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:42.298635 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:42.298780 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:42.299414 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "a8ca57412ce84fcc83b4d820ab3348fe"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.302347 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20251212 08:52:42.304016 2474 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.304492 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:42.304620 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "a8ca57412ce84fcc83b4d820ab3348fe"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.304777 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:42.322717 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:42.323338 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:42.325230 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:42.325346 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.325495 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:42.325583 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.345495 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:43621
I20251212 08:52:42.345495 2544 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:43621 every 8 connection(s)
I20251212 08:52:42.347306 2545 heartbeater.cc:344] Connected to a master server at 127.0.208.62:36405
I20251212 08:52:42.347434 2545 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:42.347782 2545 heartbeater.cc:507] Master 127.0.208.62:36405 requested a full tablet report, sending...
I20251212 08:52:42.348757 2400 ts_manager.cc:194] Registered new tserver with Master: a8ca57412ce84fcc83b4d820ab3348fe (127.0.208.1:43621)
I20251212 08:52:42.348937 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002361577s
I20251212 08:52:42.349929 2400 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51460
I20251212 08:52:42.361193 2545 heartbeater.cc:499] Master 127.0.208.62:36405 was elected leader, sending a full tablet report...
I20251212 08:52:42.361411 2400 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51478:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:42.376077 2505 tablet_service.cc:1505] Processing CreateTablet for tablet 66bdae1102014df49bdfd12b8406df7e (DEFAULT_TABLE table=client-testtb [id=a7d3217790ff40db8320dac30881eb6a]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:42.376080 2504 tablet_service.cc:1505] Processing CreateTablet for tablet 68a0ab6488bc4910856af19cf9a94aa0 (DEFAULT_TABLE table=client-testtb [id=a7d3217790ff40db8320dac30881eb6a]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:42.376600 2505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 66bdae1102014df49bdfd12b8406df7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.376879 2504 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 68a0ab6488bc4910856af19cf9a94aa0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.381964 2555 tablet_bootstrap.cc:492] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe: Bootstrap starting.
I20251212 08:52:42.384145 2555 tablet_bootstrap.cc:654] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.386143 2555 tablet_bootstrap.cc:492] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe: No bootstrap required, opened a new log
I20251212 08:52:42.386284 2555 ts_tablet_manager.cc:1403] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:42.386893 2555 raft_consensus.cc:359] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } }
I20251212 08:52:42.387034 2555 raft_consensus.cc:385] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.387102 2555 raft_consensus.cc:740] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ca57412ce84fcc83b4d820ab3348fe, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.387316 2555 consensus_queue.cc:260] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } }
I20251212 08:52:42.387451 2555 raft_consensus.cc:399] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:42.387539 2555 raft_consensus.cc:493] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:42.387651 2555 raft_consensus.cc:3060] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.389693 2555 raft_consensus.cc:515] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } }
I20251212 08:52:42.389885 2555 leader_election.cc:304] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8ca57412ce84fcc83b4d820ab3348fe; no voters:
I20251212 08:52:42.390295 2555 leader_election.cc:290] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:42.390450 2557 raft_consensus.cc:2804] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:42.390973 2555 ts_tablet_manager.cc:1434] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:52:42.390980 2557 raft_consensus.cc:697] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 1 LEADER]: Becoming Leader. State: Replica: a8ca57412ce84fcc83b4d820ab3348fe, State: Running, Role: LEADER
I20251212 08:52:42.391247 2555 tablet_bootstrap.cc:492] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe: Bootstrap starting.
I20251212 08:52:42.391280 2557 consensus_queue.cc:237] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } }
I20251212 08:52:42.392998 2555 tablet_bootstrap.cc:654] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.393738 2400 catalog_manager.cc:5654] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe reported cstate change: term changed from 0 to 1, leader changed from <none> to a8ca57412ce84fcc83b4d820ab3348fe (127.0.208.1). New cstate: current_term: 1 leader_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:42.395555 2555 tablet_bootstrap.cc:492] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe: No bootstrap required, opened a new log
I20251212 08:52:42.395691 2555 ts_tablet_manager.cc:1403] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:42.396345 2555 raft_consensus.cc:359] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } }
I20251212 08:52:42.396531 2555 raft_consensus.cc:385] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.397185 2555 raft_consensus.cc:740] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ca57412ce84fcc83b4d820ab3348fe, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.397459 2555 consensus_queue.cc:260] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } }
I20251212 08:52:42.397622 2555 raft_consensus.cc:399] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:42.397742 2555 raft_consensus.cc:493] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:42.397851 2555 raft_consensus.cc:3060] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.400252 2555 raft_consensus.cc:515] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } }
I20251212 08:52:42.400529 2555 leader_election.cc:304] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8ca57412ce84fcc83b4d820ab3348fe; no voters:
I20251212 08:52:42.400768 2555 leader_election.cc:290] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:42.400820 2557 raft_consensus.cc:2804] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:42.401006 2557 raft_consensus.cc:697] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 1 LEADER]: Becoming Leader. State: Replica: a8ca57412ce84fcc83b4d820ab3348fe, State: Running, Role: LEADER
I20251212 08:52:42.401252 2557 consensus_queue.cc:237] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } }
I20251212 08:52:42.401510 2555 ts_tablet_manager.cc:1434] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20251212 08:52:42.403466 2400 catalog_manager.cc:5654] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe reported cstate change: term changed from 0 to 1, leader changed from <none> to a8ca57412ce84fcc83b4d820ab3348fe (127.0.208.1). New cstate: current_term: 1 leader_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ca57412ce84fcc83b4d820ab3348fe" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43621 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:42.436211 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:52:42.446928 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:52:42.447310 832 tablet_replica.cc:333] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe: stopping tablet replica
I20251212 08:52:42.447574 832 raft_consensus.cc:2243] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:42.447764 832 raft_consensus.cc:2272] T 66bdae1102014df49bdfd12b8406df7e P a8ca57412ce84fcc83b4d820ab3348fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:42.448621 832 tablet_replica.cc:333] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe: stopping tablet replica
I20251212 08:52:42.448745 832 raft_consensus.cc:2243] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:42.448894 832 raft_consensus.cc:2272] T 68a0ab6488bc4910856af19cf9a94aa0 P a8ca57412ce84fcc83b4d820ab3348fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:42.463510 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:52:42.467813 832 master.cc:561] Master@127.0.208.62:36405 shutting down...
I20251212 08:52:42.479084 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:52:42.479460 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:52:42.479743 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9265c238c8a944b794fbfb1d6854b5f6: stopping tablet replica
I20251212 08:52:42.485350 832 master.cc:583] Master@127.0.208.62:36405 shutdown complete.
[ OK ] ClientTest.TestApplyToSessionWithoutFlushing_OpsBuffered (305 ms)
[ RUN ] ClientTest.TestAutoFlushBackgroundAndErrorCollector
I20251212 08:52:42.495723 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:43207
I20251212 08:52:42.496210 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:42.498459 2565 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.498754 2566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.499015 2568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:42.499292 832 server_base.cc:1047] running on GCE node
I20251212 08:52:42.499542 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:42.499595 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:42.499641 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529562499636 us; error 0 us; skew 500 ppm
I20251212 08:52:42.499795 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:42.500982 832 webserver.cc:492] Webserver started at http://127.0.208.62:40609/ using document root <none> and password file <none>
I20251212 08:52:42.501199 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:42.501268 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:42.501398 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:42.502241 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.504789 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.003s
I20251212 08:52:42.506696 2573 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.507156 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:42.507270 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.507387 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:42.531981 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:42.532614 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:42.547187 2634 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:43207 every 8 connection(s)
I20251212 08:52:42.547164 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:43207
I20251212 08:52:42.548581 2635 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.552836 2635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698: Bootstrap starting.
I20251212 08:52:42.553966 2635 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.555445 2635 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698: No bootstrap required, opened a new log
I20251212 08:52:42.556042 2635 raft_consensus.cc:359] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698" member_type: VOTER }
I20251212 08:52:42.556171 2635 raft_consensus.cc:385] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.556205 2635 raft_consensus.cc:740] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3f2c4b47cab4b1bb6e0db7a989eb698, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.556406 2635 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698" member_type: VOTER }
I20251212 08:52:42.556509 2635 raft_consensus.cc:399] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:42.556545 2635 raft_consensus.cc:493] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:42.556596 2635 raft_consensus.cc:3060] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.558377 2635 raft_consensus.cc:515] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698" member_type: VOTER }
I20251212 08:52:42.558542 2635 leader_election.cc:304] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3f2c4b47cab4b1bb6e0db7a989eb698; no voters:
I20251212 08:52:42.558899 2635 leader_election.cc:290] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:42.559034 2638 raft_consensus.cc:2804] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:42.559490 2638 raft_consensus.cc:697] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 1 LEADER]: Becoming Leader. State: Replica: a3f2c4b47cab4b1bb6e0db7a989eb698, State: Running, Role: LEADER
I20251212 08:52:42.559728 2635 sys_catalog.cc:565] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:52:42.559698 2638 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698" member_type: VOTER }
I20251212 08:52:42.560607 2639 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698" member_type: VOTER } }
I20251212 08:52:42.560653 2640 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3f2c4b47cab4b1bb6e0db7a989eb698. Latest consensus state: current_term: 1 leader_uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3f2c4b47cab4b1bb6e0db7a989eb698" member_type: VOTER } }
I20251212 08:52:42.560801 2639 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:42.560822 2640 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [sys.catalog]: This master's current role is: LEADER
I20251212 08:52:42.564556 2643 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:52:42.564886 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:52:42.565903 2643 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:52:42.568990 2643 catalog_manager.cc:1357] Generated new cluster ID: e0c150735fc14d6ab601fc082c5e5d55
I20251212 08:52:42.569090 2643 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:52:42.586127 2643 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:52:42.586957 2643 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:52:42.595213 2643 catalog_manager.cc:6027] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698: Generated new TSK 0
I20251212 08:52:42.595494 2643 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:52:42.597729 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:42.600376 2656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.600447 2657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.600767 2659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:42.601068 832 server_base.cc:1047] running on GCE node
I20251212 08:52:42.601336 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:42.601420 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:42.601480 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529562601462 us; error 0 us; skew 500 ppm
I20251212 08:52:42.601666 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:42.602978 832 webserver.cc:492] Webserver started at http://127.0.208.1:40037/ using document root <none> and password file <none>
I20251212 08:52:42.603209 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:42.603292 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:42.603451 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:42.604123 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "7cab108f9abb4e1c973338419932d96c"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.606674 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251212 08:52:42.608520 2664 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.608976 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:52:42.609119 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "7cab108f9abb4e1c973338419932d96c"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.609244 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:42.619938 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:42.620530 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:42.622177 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:42.622269 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.622413 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:42.622819 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.638274 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35409
I20251212 08:52:42.638329 2734 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35409 every 8 connection(s)
I20251212 08:52:42.640270 2735 heartbeater.cc:344] Connected to a master server at 127.0.208.62:43207
I20251212 08:52:42.640455 2735 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:42.640867 2735 heartbeater.cc:507] Master 127.0.208.62:43207 requested a full tablet report, sending...
I20251212 08:52:42.642076 2590 ts_manager.cc:194] Registered new tserver with Master: 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:42.642776 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.003455216s
I20251212 08:52:42.643586 2590 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59384
I20251212 08:52:42.654860 2735 heartbeater.cc:499] Master 127.0.208.62:43207 was elected leader, sending a full tablet report...
I20251212 08:52:42.656219 2590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:52:42.671933 2694 tablet_service.cc:1505] Processing CreateTablet for tablet e2f4303ab1754ce58f47f39d80e9c38c (DEFAULT_TABLE table=client-testtb [id=d22a95a7acfc4d14a66ba766859cf11a]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:52:42.671972 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 0fd6c689a671492194b6e799e903bfe0 (DEFAULT_TABLE table=client-testtb [id=d22a95a7acfc4d14a66ba766859cf11a]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:52:42.672588 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fd6c689a671492194b6e799e903bfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.672973 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2f4303ab1754ce58f47f39d80e9c38c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.678015 2745 tablet_bootstrap.cc:492] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:42.679687 2745 tablet_bootstrap.cc:654] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.681399 2745 tablet_bootstrap.cc:492] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:42.681543 2745 ts_tablet_manager.cc:1403] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:42.682224 2745 raft_consensus.cc:359] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.682407 2745 raft_consensus.cc:385] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.682524 2745 raft_consensus.cc:740] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.682755 2745 consensus_queue.cc:260] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.682912 2745 raft_consensus.cc:399] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:42.682993 2745 raft_consensus.cc:493] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:42.683104 2745 raft_consensus.cc:3060] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.685490 2745 raft_consensus.cc:515] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.685750 2745 leader_election.cc:304] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:42.686203 2745 leader_election.cc:290] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:42.686367 2747 raft_consensus.cc:2804] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:42.686880 2745 ts_tablet_manager.cc:1434] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20251212 08:52:42.687144 2745 tablet_bootstrap.cc:492] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:42.688515 2747 raft_consensus.cc:697] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:52:42.688755 2745 tablet_bootstrap.cc:654] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.688835 2747 consensus_queue.cc:237] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.690671 2745 tablet_bootstrap.cc:492] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:42.690865 2745 ts_tablet_manager.cc:1403] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:42.691129 2590 catalog_manager.cc:5654] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:42.691481 2745 raft_consensus.cc:359] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.691617 2745 raft_consensus.cc:385] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.691705 2745 raft_consensus.cc:740] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.691933 2745 consensus_queue.cc:260] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.692058 2745 raft_consensus.cc:399] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:52:42.692160 2745 raft_consensus.cc:493] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:52:42.692260 2745 raft_consensus.cc:3060] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.694295 2745 raft_consensus.cc:515] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.694504 2745 leader_election.cc:304] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:42.694715 2745 leader_election.cc:290] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:52:42.694762 2747 raft_consensus.cc:2804] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:42.694973 2747 raft_consensus.cc:697] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:52:42.695206 2745 ts_tablet_manager.cc:1434] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20251212 08:52:42.695259 2747 consensus_queue.cc:237] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.698133 2590 catalog_manager.cc:5654] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:42.707479 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:52:42.710232 2754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.710520 2755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.711154 2757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:42.711928 832 server_base.cc:1047] running on GCE node
I20251212 08:52:42.712256 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:42.712373 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:42.712448 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529562712408 us; error 0 us; skew 500 ppm
I20251212 08:52:42.712669 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:42.713883 832 webserver.cc:492] Webserver started at http://127.0.208.2:41713/ using document root <none> and password file <none>
I20251212 08:52:42.714112 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:42.714212 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:42.714356 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:42.715019 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-1-root/instance:
uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.717849 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20251212 08:52:42.719585 2762 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.720033 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:52:42.720136 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.720263 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:42.740741 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:42.741768 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:42.743616 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:42.743731 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.743880 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:42.744004 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.767620 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:41879
I20251212 08:52:42.767699 2831 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:41879 every 8 connection(s)
I20251212 08:52:42.769418 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251212 08:52:42.769656 2832 heartbeater.cc:344] Connected to a master server at 127.0.208.62:43207
I20251212 08:52:42.769795 2832 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:42.770282 2832 heartbeater.cc:507] Master 127.0.208.62:43207 requested a full tablet report, sending...
I20251212 08:52:42.771448 2590 ts_manager.cc:194] Registered new tserver with Master: 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
W20251212 08:52:42.773435 2834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:52:42.773751 2835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:42.774011 2590 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59398
W20251212 08:52:42.774716 2837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:52:42.776368 832 server_base.cc:1047] running on GCE node
I20251212 08:52:42.776649 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:52:42.776712 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:52:42.776760 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529562776734 us; error 0 us; skew 500 ppm
I20251212 08:52:42.776885 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:52:42.778600 832 webserver.cc:492] Webserver started at http://127.0.208.3:33539/ using document root <none> and password file <none>
I20251212 08:52:42.778813 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:52:42.778898 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:52:42.779002 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:52:42.779592 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-2-root/instance:
uuid: "2d41c990aa9f453e96ec315fab508132"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.782181 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:52:42.784029 2842 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.784657 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:52:42.784798 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "2d41c990aa9f453e96ec315fab508132"
format_stamp: "Formatted at 2025-12-12 08:52:42 on dist-test-slave-qdpb"
I20251212 08:52:42.784965 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestAutoFlushBackgroundAndErrorCollector.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:52:42.785341 2832 heartbeater.cc:499] Master 127.0.208.62:43207 was elected leader, sending a full tablet report...
I20251212 08:52:42.800738 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:52:42.801304 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:52:42.803134 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:52:42.803274 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.803416 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:52:42.803548 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:52:42.831377 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:33005
I20251212 08:52:42.831422 2911 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:33005 every 8 connection(s)
I20251212 08:52:42.833177 2912 heartbeater.cc:344] Connected to a master server at 127.0.208.62:43207
I20251212 08:52:42.833309 2912 heartbeater.cc:461] Registering TS with master...
I20251212 08:52:42.833750 2912 heartbeater.cc:507] Master 127.0.208.62:43207 requested a full tablet report, sending...
I20251212 08:52:42.834595 2589 ts_manager.cc:194] Registered new tserver with Master: 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:42.834659 832 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.002353197s
I20251212 08:52:42.835769 2589 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59422
I20251212 08:52:42.836743 2588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.0"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:42.837634 2588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.0 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:42.847103 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 47b760e1902f4f5f84cdb54634e6da97 (DEFAULT_TABLE table=table.0 [id=3f1bb6e57a7b416d96116ae3823d69a2]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:42.848783 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b760e1902f4f5f84cdb54634e6da97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.849462 2912 heartbeater.cc:499] Master 127.0.208.62:43207 was elected leader, sending a full tablet report...
I20251212 08:52:42.850193 2695 tablet_service.cc:1505] Processing CreateTablet for tablet a9cc0c9708724db590f46d357c6a3639 (DEFAULT_TABLE table=table.0 [id=3f1bb6e57a7b416d96116ae3823d69a2]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:42.851603 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc0c9708724db590f46d357c6a3639. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.867466 2745 tablet_bootstrap.cc:492] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:42.870994 2745 tablet_bootstrap.cc:654] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.876264 2745 tablet_bootstrap.cc:492] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:42.876621 2745 ts_tablet_manager.cc:1403] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.004s
I20251212 08:52:42.877487 2745 raft_consensus.cc:359] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.877827 2745 raft_consensus.cc:385] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.878017 2745 raft_consensus.cc:740] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.878346 2745 consensus_queue.cc:260] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.878970 2745 ts_tablet_manager.cc:1434] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:52:42.879436 2745 tablet_bootstrap.cc:492] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:42.880743 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 47b760e1902f4f5f84cdb54634e6da97 (DEFAULT_TABLE table=table.0 [id=3f1bb6e57a7b416d96116ae3823d69a2]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:42.881383 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b760e1902f4f5f84cdb54634e6da97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.881750 2745 tablet_bootstrap.cc:654] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.881810 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 47b760e1902f4f5f84cdb54634e6da97 (DEFAULT_TABLE table=table.0 [id=3f1bb6e57a7b416d96116ae3823d69a2]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:42.882804 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b760e1902f4f5f84cdb54634e6da97. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.885748 2793 tablet_service.cc:1505] Processing CreateTablet for tablet a9cc0c9708724db590f46d357c6a3639 (DEFAULT_TABLE table=table.0 [id=3f1bb6e57a7b416d96116ae3823d69a2]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:42.887320 2873 tablet_service.cc:1505] Processing CreateTablet for tablet a9cc0c9708724db590f46d357c6a3639 (DEFAULT_TABLE table=table.0 [id=3f1bb6e57a7b416d96116ae3823d69a2]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:42.888743 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc0c9708724db590f46d357c6a3639. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:42.889035 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9cc0c9708724db590f46d357c6a3639. 1 dirs total, 0 dirs full, 0 dirs failed
W20251212 08:52:42.889801 2736 tablet.cc:2378] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251212 08:52:42.890791 2745 tablet_bootstrap.cc:492] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:42.891007 2745 ts_tablet_manager.cc:1403] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20251212 08:52:42.891942 2745 raft_consensus.cc:359] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:42.892185 2745 raft_consensus.cc:385] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.892321 2745 raft_consensus.cc:740] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.892728 2745 consensus_queue.cc:260] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:42.897330 2745 ts_tablet_manager.cc:1434] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20251212 08:52:42.899761 2919 tablet_bootstrap.cc:492] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:42.909556 2920 tablet_bootstrap.cc:492] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:42.911602 2919 tablet_bootstrap.cc:654] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.912351 2920 tablet_bootstrap.cc:654] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.915910 2919 tablet_bootstrap.cc:492] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:42.915947 2920 tablet_bootstrap.cc:492] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:42.916155 2919 ts_tablet_manager.cc:1403] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.010s
I20251212 08:52:42.916153 2920 ts_tablet_manager.cc:1403] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.001s
I20251212 08:52:42.917063 2919 raft_consensus.cc:359] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:42.917083 2920 raft_consensus.cc:359] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.917241 2919 raft_consensus.cc:385] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.917271 2920 raft_consensus.cc:385] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.917346 2919 raft_consensus.cc:740] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.917382 2920 raft_consensus.cc:740] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.917590 2919 consensus_queue.cc:260] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:42.917718 2920 consensus_queue.cc:260] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.918498 2919 ts_tablet_manager.cc:1434] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251212 08:52:42.918601 2920 ts_tablet_manager.cc:1434] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:42.918792 2919 tablet_bootstrap.cc:492] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:42.918906 2920 tablet_bootstrap.cc:492] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:42.920341 2747 raft_consensus.cc:493] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:42.920526 2747 raft_consensus.cc:515] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:42.921002 2920 tablet_bootstrap.cc:654] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.921056 2919 tablet_bootstrap.cc:654] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:42.922494 2747 leader_election.cc:290] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:42.934352 2919 tablet_bootstrap.cc:492] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:42.934612 2919 ts_tablet_manager.cc:1403] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.010s
I20251212 08:52:42.935041 2920 tablet_bootstrap.cc:492] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:42.935302 2920 ts_tablet_manager.cc:1403] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.001s
I20251212 08:52:42.935695 2919 raft_consensus.cc:359] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.935957 2919 raft_consensus.cc:385] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.936053 2919 raft_consensus.cc:740] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.936378 2919 consensus_queue.cc:260] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:42.936661 2920 raft_consensus.cc:359] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:42.936937 2920 raft_consensus.cc:385] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:42.937077 2920 raft_consensus.cc:740] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:42.937114 2919 ts_tablet_manager.cc:1434] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251212 08:52:42.937222 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9cc0c9708724db590f46d357c6a3639" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:42.937515 2920 consensus_queue.cc:260] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:42.937803 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9cc0c9708724db590f46d357c6a3639" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:42.938176 2887 raft_consensus.cc:2468] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:42.938552 2807 raft_consensus.cc:2468] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:42.938756 2920 ts_tablet_manager.cc:1434] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:52:42.938863 2667 leader_election.cc:304] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:42.939289 2747 raft_consensus.cc:2804] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:42.939424 2747 raft_consensus.cc:493] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:42.939545 2747 raft_consensus.cc:3060] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.943866 2747 raft_consensus.cc:515] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:42.944564 2747 leader_election.cc:290] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:42.944918 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9cc0c9708724db590f46d357c6a3639" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:42.945029 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9cc0c9708724db590f46d357c6a3639" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:42.945104 2807 raft_consensus.cc:3060] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.945225 2887 raft_consensus.cc:3060] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:42.947889 2887 raft_consensus.cc:2468] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:42.947901 2807 raft_consensus.cc:2468] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:42.948486 2667 leader_election.cc:304] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:42.948876 2747 raft_consensus.cc:2804] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:42.949038 2747 raft_consensus.cc:697] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:52:42.949342 2747 consensus_queue.cc:237] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:42.952667 2587 catalog_manager.cc:5654] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:43.004015 2924 raft_consensus.cc:493] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:43.004734 2924 raft_consensus.cc:515] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:43.005652 2924 leader_election.cc:290] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:43.012867 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47b760e1902f4f5f84cdb54634e6da97" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:43.013275 2705 raft_consensus.cc:2468] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:43.013825 2763 leader_election.cc:304] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:43.014207 2924 raft_consensus.cc:2804] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:43.014339 2924 raft_consensus.cc:493] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:43.014446 2924 raft_consensus.cc:3060] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:43.014825 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47b760e1902f4f5f84cdb54634e6da97" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:43.015153 2887 raft_consensus.cc:2468] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:43.017056 2924 raft_consensus.cc:515] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:43.017643 2924 leader_election.cc:290] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:43.018150 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47b760e1902f4f5f84cdb54634e6da97" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:43.018162 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47b760e1902f4f5f84cdb54634e6da97" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:43.018380 2705 raft_consensus.cc:3060] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:43.018389 2887 raft_consensus.cc:3060] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:43.022214 2705 raft_consensus.cc:2468] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:43.022212 2887 raft_consensus.cc:2468] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:43.022704 2763 leader_election.cc:304] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:43.023020 2924 raft_consensus.cc:2804] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:43.023404 2924 raft_consensus.cc:697] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:43.023672 2924 consensus_queue.cc:237] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:43.026338 2590 catalog_manager.cc:5654] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:43.073992 2887 raft_consensus.cc:1275] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:43.074368 2886 raft_consensus.cc:1275] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:43.074486 2807 raft_consensus.cc:1275] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:43.074019 2705 raft_consensus.cc:1275] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:43.075212 2924 consensus_queue.cc:1048] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:43.075652 2930 consensus_queue.cc:1048] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:43.075237 2748 consensus_queue.cc:1048] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:43.076938 2747 consensus_queue.cc:1048] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:43.092520 2752 mvcc.cc:204] Tried to move back new op lower bound from 7231609090343493632 to 7231609090147348480. Current Snapshot: MvccSnapshot[applied={T|T < 7231609090343493632}]
W20251212 08:52:43.330271 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 174ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:43.586231 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 433ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:43.844374 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 689ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:44.099128 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00159e+06us
I20251212 08:52:44.100167 2587 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.1"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:44.101190 2587 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:44.111644 2795 tablet_service.cc:1505] Processing CreateTablet for tablet adaf3ccf3aa645bcb2064ce429ac8ee4 (DEFAULT_TABLE table=table.1 [id=0558a3ea33434d468920b2db45a31c5a]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:44.111944 2876 tablet_service.cc:1505] Processing CreateTablet for tablet adaf3ccf3aa645bcb2064ce429ac8ee4 (DEFAULT_TABLE table=table.1 [id=0558a3ea33434d468920b2db45a31c5a]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:44.112092 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adaf3ccf3aa645bcb2064ce429ac8ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:44.112247 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adaf3ccf3aa645bcb2064ce429ac8ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:44.117092 2695 tablet_service.cc:1505] Processing CreateTablet for tablet adaf3ccf3aa645bcb2064ce429ac8ee4 (DEFAULT_TABLE table=table.1 [id=0558a3ea33434d468920b2db45a31c5a]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:44.117136 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 0e1c7c8118304721a5c37409bb220623 (DEFAULT_TABLE table=table.1 [id=0558a3ea33434d468920b2db45a31c5a]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:44.117467 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adaf3ccf3aa645bcb2064ce429ac8ee4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:44.118865 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e1c7c8118304721a5c37409bb220623. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:44.121698 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 0e1c7c8118304721a5c37409bb220623 (DEFAULT_TABLE table=table.1 [id=0558a3ea33434d468920b2db45a31c5a]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:44.122071 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e1c7c8118304721a5c37409bb220623. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:44.125015 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 0e1c7c8118304721a5c37409bb220623 (DEFAULT_TABLE table=table.1 [id=0558a3ea33434d468920b2db45a31c5a]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:44.125459 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0e1c7c8118304721a5c37409bb220623. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:44.128638 2955 tablet_bootstrap.cc:492] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:44.130249 2955 tablet_bootstrap.cc:654] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:44.132215 2957 tablet_bootstrap.cc:492] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:44.133693 2957 tablet_bootstrap.cc:654] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:44.137988 2958 tablet_bootstrap.cc:492] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:44.139407 2958 tablet_bootstrap.cc:654] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:44.141392 2955 tablet_bootstrap.cc:492] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:44.141510 2955 ts_tablet_manager.cc:1403] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.000s
I20251212 08:52:44.141891 2957 tablet_bootstrap.cc:492] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:44.141995 2957 ts_tablet_manager.cc:1403] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20251212 08:52:44.142243 2955 raft_consensus.cc:359] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:44.142401 2955 raft_consensus.cc:385] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:44.142452 2955 raft_consensus.cc:740] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:44.142611 2957 raft_consensus.cc:359] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:44.142670 2955 consensus_queue.cc:260] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:44.142820 2957 raft_consensus.cc:385] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:44.142884 2957 raft_consensus.cc:740] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:44.143113 2957 consensus_queue.cc:260] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:44.143500 2957 ts_tablet_manager.cc:1434] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:44.143929 2955 ts_tablet_manager.cc:1434] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:44.144069 2958 tablet_bootstrap.cc:492] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:44.144169 2958 ts_tablet_manager.cc:1403] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20251212 08:52:44.144858 2958 raft_consensus.cc:359] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:44.145009 2958 raft_consensus.cc:385] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:44.145064 2958 raft_consensus.cc:740] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:44.145247 2958 consensus_queue.cc:260] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:44.144177 2955 tablet_bootstrap.cc:492] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:44.148712 2958 ts_tablet_manager.cc:1434] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20251212 08:52:44.148986 2957 tablet_bootstrap.cc:492] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:44.149020 2958 tablet_bootstrap.cc:492] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:44.150223 2957 tablet_bootstrap.cc:654] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:44.150303 2958 tablet_bootstrap.cc:654] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:44.156360 2955 tablet_bootstrap.cc:654] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:44.158581 2958 tablet_bootstrap.cc:492] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:44.158720 2958 ts_tablet_manager.cc:1403] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20251212 08:52:44.159422 2958 raft_consensus.cc:359] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:44.159587 2958 raft_consensus.cc:385] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:44.159638 2958 raft_consensus.cc:740] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:44.159837 2958 consensus_queue.cc:260] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:44.160256 2958 ts_tablet_manager.cc:1434] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251212 08:52:44.161039 2957 tablet_bootstrap.cc:492] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:44.161201 2957 ts_tablet_manager.cc:1403] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.006s
I20251212 08:52:44.161501 2955 tablet_bootstrap.cc:492] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:44.161602 2955 ts_tablet_manager.cc:1403] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20251212 08:52:44.161918 2957 raft_consensus.cc:359] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:44.162066 2957 raft_consensus.cc:385] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:44.162113 2957 raft_consensus.cc:740] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:44.162142 2955 raft_consensus.cc:359] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:44.162313 2955 raft_consensus.cc:385] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:44.162290 2957 consensus_queue.cc:260] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:44.162369 2955 raft_consensus.cc:740] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:44.162575 2955 consensus_queue.cc:260] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:44.163024 2955 ts_tablet_manager.cc:1434] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251212 08:52:44.163195 2957 ts_tablet_manager.cc:1434] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:52:44.178910 2924 raft_consensus.cc:493] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:44.179072 2924 raft_consensus.cc:515] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:44.179664 2924 leader_election.cc:290] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:44.179992 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e1c7c8118304721a5c37409bb220623" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:44.180094 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e1c7c8118304721a5c37409bb220623" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:44.180163 2705 raft_consensus.cc:2468] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:44.180217 2886 raft_consensus.cc:2468] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:44.180575 2765 leader_election.cc:304] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:44.180823 2924 raft_consensus.cc:2804] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:44.180881 2924 raft_consensus.cc:493] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:44.180922 2924 raft_consensus.cc:3060] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:44.182766 2924 raft_consensus.cc:515] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:44.183414 2924 leader_election.cc:290] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:44.184011 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e1c7c8118304721a5c37409bb220623" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:44.184123 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0e1c7c8118304721a5c37409bb220623" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:44.184175 2705 raft_consensus.cc:3060] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:44.184234 2886 raft_consensus.cc:3060] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:44.187490 2705 raft_consensus.cc:2468] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:44.187490 2886 raft_consensus.cc:2468] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:44.187984 2763 leader_election.cc:304] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:44.188239 2924 raft_consensus.cc:493] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:44.188403 2924 raft_consensus.cc:515] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:44.189209 2965 raft_consensus.cc:2804] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:44.190616 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "adaf3ccf3aa645bcb2064ce429ac8ee4" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:44.190800 2886 raft_consensus.cc:2468] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:44.191152 2765 leader_election.cc:304] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:44.192130 2924 leader_election.cc:290] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:44.192332 2924 raft_consensus.cc:2804] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:44.192399 2924 raft_consensus.cc:493] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:44.192452 2924 raft_consensus.cc:3060] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:44.195264 2924 raft_consensus.cc:515] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:44.195869 2924 leader_election.cc:290] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:44.196834 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "adaf3ccf3aa645bcb2064ce429ac8ee4" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:44.197080 2704 raft_consensus.cc:3060] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:44.201390 2704 raft_consensus.cc:2468] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:44.204075 2763 leader_election.cc:304] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:44.206002 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "adaf3ccf3aa645bcb2064ce429ac8ee4" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:44.206143 2967 raft_consensus.cc:2804] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:44.206269 2967 raft_consensus.cc:697] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:44.206300 2705 raft_consensus.cc:2376] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1. Re-sending same reply.
I20251212 08:52:44.206549 2967 consensus_queue.cc:237] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:44.207602 2965 raft_consensus.cc:697] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:44.207634 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "adaf3ccf3aa645bcb2064ce429ac8ee4" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:44.207791 2886 raft_consensus.cc:3060] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:44.207875 2965 consensus_queue.cc:237] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:44.210527 2886 raft_consensus.cc:2468] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:44.214156 2590 catalog_manager.cc:5654] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:44.218760 2590 catalog_manager.cc:5654] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:44.236358 2886 raft_consensus.cc:1275] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:44.236359 2705 raft_consensus.cc:1275] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:44.237111 2967 consensus_queue.cc:1048] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:44.237288 2965 consensus_queue.cc:1048] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:44.238485 2887 raft_consensus.cc:1275] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:44.238891 2966 consensus_queue.cc:1048] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:44.239192 2704 raft_consensus.cc:1275] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:44.241467 2966 consensus_queue.cc:1048] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:52:44.562124 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 250ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:44.817227 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 509ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:45.070600 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 764ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:45.254384 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00093e+06us
I20251212 08:52:45.256016 2590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.2"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:45.256776 2590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:45.265709 2793 tablet_service.cc:1505] Processing CreateTablet for tablet ab43cb77736b4eaf9dd5c6bd73cfc733 (DEFAULT_TABLE table=table.2 [id=23911c5e10bf44cab0edf50e31d8fbad]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:45.266018 2873 tablet_service.cc:1505] Processing CreateTablet for tablet ab43cb77736b4eaf9dd5c6bd73cfc733 (DEFAULT_TABLE table=table.2 [id=23911c5e10bf44cab0edf50e31d8fbad]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:45.266223 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab43cb77736b4eaf9dd5c6bd73cfc733. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:45.266366 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab43cb77736b4eaf9dd5c6bd73cfc733. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:45.267078 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 701bc5e1a89b4e10bf2d468cf0c0e783 (DEFAULT_TABLE table=table.2 [id=23911c5e10bf44cab0edf50e31d8fbad]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:45.267468 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701bc5e1a89b4e10bf2d468cf0c0e783. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:45.267519 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 701bc5e1a89b4e10bf2d468cf0c0e783 (DEFAULT_TABLE table=table.2 [id=23911c5e10bf44cab0edf50e31d8fbad]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:45.267885 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701bc5e1a89b4e10bf2d468cf0c0e783. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:45.269153 2694 tablet_service.cc:1505] Processing CreateTablet for tablet ab43cb77736b4eaf9dd5c6bd73cfc733 (DEFAULT_TABLE table=table.2 [id=23911c5e10bf44cab0edf50e31d8fbad]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:45.269536 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab43cb77736b4eaf9dd5c6bd73cfc733. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:45.276363 3000 tablet_bootstrap.cc:492] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:45.277264 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 701bc5e1a89b4e10bf2d468cf0c0e783 (DEFAULT_TABLE table=table.2 [id=23911c5e10bf44cab0edf50e31d8fbad]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:45.277679 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 701bc5e1a89b4e10bf2d468cf0c0e783. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:45.278462 3000 tablet_bootstrap.cc:654] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:45.281467 3000 tablet_bootstrap.cc:492] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:45.281630 3000 ts_tablet_manager.cc:1403] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251212 08:52:45.282553 3000 raft_consensus.cc:359] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:45.282775 3000 raft_consensus.cc:385] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:45.286126 3000 raft_consensus.cc:740] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:45.286800 3000 consensus_queue.cc:260] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:45.290040 3000 ts_tablet_manager.cc:1434] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20251212 08:52:45.290339 3000 tablet_bootstrap.cc:492] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:45.292549 3000 tablet_bootstrap.cc:654] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:45.293460 2998 tablet_bootstrap.cc:492] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:45.294776 2998 tablet_bootstrap.cc:654] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:45.295581 2999 tablet_bootstrap.cc:492] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:45.296658 3000 tablet_bootstrap.cc:492] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:45.296883 3000 ts_tablet_manager.cc:1403] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251212 08:52:45.297446 2999 tablet_bootstrap.cc:654] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:45.297653 3000 raft_consensus.cc:359] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:45.297924 3000 raft_consensus.cc:385] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:45.298019 3000 raft_consensus.cc:740] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:45.298305 3000 consensus_queue.cc:260] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:45.299088 3000 ts_tablet_manager.cc:1434] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:52:45.299468 2998 tablet_bootstrap.cc:492] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:45.299612 2998 ts_tablet_manager.cc:1403] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.000s
I20251212 08:52:45.300239 2999 tablet_bootstrap.cc:492] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:45.300429 2999 ts_tablet_manager.cc:1403] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.004s
I20251212 08:52:45.300239 2998 raft_consensus.cc:359] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:45.300561 2998 raft_consensus.cc:385] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:45.300671 2998 raft_consensus.cc:740] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:45.300938 2998 consensus_queue.cc:260] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:45.301170 2999 raft_consensus.cc:359] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:45.302484 2999 raft_consensus.cc:385] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:45.302605 2999 raft_consensus.cc:740] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:45.302853 2999 consensus_queue.cc:260] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:45.303471 2999 ts_tablet_manager.cc:1434] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:52:45.304765 2999 tablet_bootstrap.cc:492] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:45.306823 2999 tablet_bootstrap.cc:654] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:45.308717 2998 ts_tablet_manager.cc:1434] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20251212 08:52:45.309017 2998 tablet_bootstrap.cc:492] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:45.309201 2999 tablet_bootstrap.cc:492] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:45.309348 2999 ts_tablet_manager.cc:1403] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20251212 08:52:45.309904 2999 raft_consensus.cc:359] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:45.310076 2999 raft_consensus.cc:385] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:45.310164 2999 raft_consensus.cc:740] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:45.310400 2999 consensus_queue.cc:260] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:45.310937 2999 ts_tablet_manager.cc:1434] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20251212 08:52:45.310935 2998 tablet_bootstrap.cc:654] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:45.312999 2998 tablet_bootstrap.cc:492] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:45.313189 2998 ts_tablet_manager.cc:1403] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:45.313849 2998 raft_consensus.cc:359] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:45.314047 2998 raft_consensus.cc:385] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:45.314144 2998 raft_consensus.cc:740] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:45.314401 2998 consensus_queue.cc:260] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:45.314922 2998 ts_tablet_manager.cc:1434] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251212 08:52:45.321619 2924 raft_consensus.cc:493] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:45.321787 2924 raft_consensus.cc:515] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:45.322392 2924 leader_election.cc:290] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:45.322857 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab43cb77736b4eaf9dd5c6bd73cfc733" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:45.322959 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab43cb77736b4eaf9dd5c6bd73cfc733" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:45.323038 2886 raft_consensus.cc:2468] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:45.323210 2705 raft_consensus.cc:2468] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:45.323554 2765 leader_election.cc:304] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:45.323952 2924 raft_consensus.cc:2804] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:45.324077 2924 raft_consensus.cc:493] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:45.324180 2924 raft_consensus.cc:3060] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:45.326375 2924 raft_consensus.cc:515] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:45.327024 2924 leader_election.cc:290] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:45.327507 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab43cb77736b4eaf9dd5c6bd73cfc733" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:45.327530 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ab43cb77736b4eaf9dd5c6bd73cfc733" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:45.327739 2886 raft_consensus.cc:3060] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:45.327744 2705 raft_consensus.cc:3060] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:45.331049 2886 raft_consensus.cc:2468] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:45.331317 2705 raft_consensus.cc:2468] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:45.331483 2765 leader_election.cc:304] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:45.331787 2924 raft_consensus.cc:2804] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:45.332154 2924 raft_consensus.cc:697] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:45.332446 2924 consensus_queue.cc:237] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:45.334957 2586 catalog_manager.cc:5654] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:45.411309 2924 raft_consensus.cc:493] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:45.411556 2924 raft_consensus.cc:515] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:45.412516 2924 leader_election.cc:290] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:45.412631 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "701bc5e1a89b4e10bf2d468cf0c0e783" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:45.412827 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "701bc5e1a89b4e10bf2d468cf0c0e783" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:45.412885 2705 raft_consensus.cc:2468] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:45.412986 2886 raft_consensus.cc:2468] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:45.413323 2765 leader_election.cc:304] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:45.413651 2924 raft_consensus.cc:2804] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:45.413759 2924 raft_consensus.cc:493] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:45.413892 2924 raft_consensus.cc:3060] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:45.415958 2924 raft_consensus.cc:515] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:45.416592 2924 leader_election.cc:290] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:45.417048 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "701bc5e1a89b4e10bf2d468cf0c0e783" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:45.417053 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "701bc5e1a89b4e10bf2d468cf0c0e783" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:45.417263 2705 raft_consensus.cc:3060] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:45.417296 2886 raft_consensus.cc:3060] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:45.419349 2886 raft_consensus.cc:2468] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:45.419353 2705 raft_consensus.cc:2468] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:45.419795 2765 leader_election.cc:304] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:45.420090 2924 raft_consensus.cc:2804] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:45.420274 2924 raft_consensus.cc:697] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:45.420607 2924 consensus_queue.cc:237] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:45.423538 2586 catalog_manager.cc:5654] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:45.444818 2705 raft_consensus.cc:1275] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:45.445188 2704 raft_consensus.cc:1275] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:45.445587 2886 raft_consensus.cc:1275] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:45.445943 2924 consensus_queue.cc:1048] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:45.446139 2886 raft_consensus.cc:1275] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:45.446033 3008 consensus_queue.cc:1048] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:45.446280 3013 consensus_queue.cc:1048] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251212 08:52:45.446702 3008 consensus_queue.cc:1048] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:45.456550 3012 mvcc.cc:204] Tried to move back new op lower bound from 7231609100057272320 to 7231609099603767296. Current Snapshot: MvccSnapshot[applied={T|T < 7231609100057272320}]
W20251212 08:52:45.727384 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 215ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:45.985077 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 471ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:46.241648 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 729ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:46.457522 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00115e+06us
I20251212 08:52:46.459051 2587 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.3"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:46.460089 2587 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.3 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:46.472357 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 8f66b4c1169b4621838e302888e31047 (DEFAULT_TABLE table=table.3 [id=d3e4a95d693d4f6ea189178e028a9a68]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:46.472854 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f66b4c1169b4621838e302888e31047. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:46.473042 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 8f66b4c1169b4621838e302888e31047 (DEFAULT_TABLE table=table.3 [id=d3e4a95d693d4f6ea189178e028a9a68]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:46.473455 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f66b4c1169b4621838e302888e31047. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:46.476692 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 8f66b4c1169b4621838e302888e31047 (DEFAULT_TABLE table=table.3 [id=d3e4a95d693d4f6ea189178e028a9a68]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:46.477063 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f66b4c1169b4621838e302888e31047. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:46.481122 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 888c406ad456495cb6bb53abac986a94 (DEFAULT_TABLE table=table.3 [id=d3e4a95d693d4f6ea189178e028a9a68]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:46.481498 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888c406ad456495cb6bb53abac986a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:46.483912 3032 tablet_bootstrap.cc:492] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:46.484225 3033 tablet_bootstrap.cc:492] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:46.486229 3032 tablet_bootstrap.cc:654] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:46.486544 3033 tablet_bootstrap.cc:654] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:46.487213 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 888c406ad456495cb6bb53abac986a94 (DEFAULT_TABLE table=table.3 [id=d3e4a95d693d4f6ea189178e028a9a68]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:46.487558 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888c406ad456495cb6bb53abac986a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:46.488210 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 888c406ad456495cb6bb53abac986a94 (DEFAULT_TABLE table=table.3 [id=d3e4a95d693d4f6ea189178e028a9a68]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:46.488555 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 888c406ad456495cb6bb53abac986a94. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:46.503268 3033 tablet_bootstrap.cc:492] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:46.503546 3033 ts_tablet_manager.cc:1403] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.019s user 0.003s sys 0.006s
I20251212 08:52:46.504535 3033 raft_consensus.cc:359] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:46.504689 3036 tablet_bootstrap.cc:492] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:46.504854 3033 raft_consensus.cc:385] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:46.505532 3033 raft_consensus.cc:740] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:46.505828 3033 consensus_queue.cc:260] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:46.506482 3033 ts_tablet_manager.cc:1434] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:52:46.506778 3033 tablet_bootstrap.cc:492] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:46.506959 3036 tablet_bootstrap.cc:654] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:46.508347 3033 tablet_bootstrap.cc:654] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:46.508745 3032 tablet_bootstrap.cc:492] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:46.508939 3032 ts_tablet_manager.cc:1403] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.020s
I20251212 08:52:46.509675 3032 raft_consensus.cc:359] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:46.509948 3032 raft_consensus.cc:385] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:46.510051 3032 raft_consensus.cc:740] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:46.510313 3032 consensus_queue.cc:260] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:46.511823 3032 ts_tablet_manager.cc:1434] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:52:46.512058 3032 tablet_bootstrap.cc:492] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:46.512336 3033 tablet_bootstrap.cc:492] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:46.512449 3033 ts_tablet_manager.cc:1403] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20251212 08:52:46.513070 3033 raft_consensus.cc:359] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:46.513221 3033 raft_consensus.cc:385] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:46.513279 3033 raft_consensus.cc:740] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:46.513371 3032 tablet_bootstrap.cc:654] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:46.513465 3033 consensus_queue.cc:260] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:46.514123 3033 ts_tablet_manager.cc:1434] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:46.515046 3036 tablet_bootstrap.cc:492] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:46.515218 3036 ts_tablet_manager.cc:1403] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.000s
I20251212 08:52:46.516041 3036 raft_consensus.cc:359] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:46.516227 3036 raft_consensus.cc:385] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:46.516314 3036 raft_consensus.cc:740] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:46.516601 3032 tablet_bootstrap.cc:492] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:46.516568 3036 consensus_queue.cc:260] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:46.516835 3032 ts_tablet_manager.cc:1403] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251212 08:52:46.517577 3032 raft_consensus.cc:359] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:46.517783 3032 raft_consensus.cc:385] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:46.517885 3032 raft_consensus.cc:740] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:46.518162 3032 consensus_queue.cc:260] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:46.518437 3036 ts_tablet_manager.cc:1434] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20251212 08:52:46.518716 3036 tablet_bootstrap.cc:492] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:46.519311 3032 ts_tablet_manager.cc:1434] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:46.520375 3036 tablet_bootstrap.cc:654] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:46.524394 3036 tablet_bootstrap.cc:492] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:46.524537 3036 ts_tablet_manager.cc:1403] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20251212 08:52:46.525307 3036 raft_consensus.cc:359] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:46.525509 3036 raft_consensus.cc:385] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:46.525586 3036 raft_consensus.cc:740] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:46.526585 3036 consensus_queue.cc:260] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:46.527143 3036 ts_tablet_manager.cc:1434] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251212 08:52:46.540724 2954 raft_consensus.cc:493] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:46.540877 2954 raft_consensus.cc:515] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:46.541538 2954 leader_election.cc:290] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:46.541909 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "888c406ad456495cb6bb53abac986a94" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:46.542073 2886 raft_consensus.cc:2468] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:46.542110 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "888c406ad456495cb6bb53abac986a94" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:46.542284 2807 raft_consensus.cc:2468] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:46.542436 2667 leader_election.cc:304] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:46.542672 2954 raft_consensus.cc:2804] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:46.542738 2954 raft_consensus.cc:493] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:46.542799 2954 raft_consensus.cc:3060] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:46.544765 2954 raft_consensus.cc:515] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:46.545317 2954 leader_election.cc:290] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:46.545691 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "888c406ad456495cb6bb53abac986a94" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:46.545691 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "888c406ad456495cb6bb53abac986a94" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:46.545861 2886 raft_consensus.cc:3060] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:46.545895 2807 raft_consensus.cc:3060] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:46.547886 2886 raft_consensus.cc:2468] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:46.547884 2807 raft_consensus.cc:2468] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:46.548260 2667 leader_election.cc:304] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:46.548512 2954 raft_consensus.cc:2804] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:46.548938 2954 raft_consensus.cc:697] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:52:46.549156 2954 consensus_queue.cc:237] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:46.551363 2587 catalog_manager.cc:5654] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:46.688505 3040 raft_consensus.cc:493] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:46.688695 3040 raft_consensus.cc:515] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:46.689411 3040 leader_election.cc:290] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:46.696324 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8f66b4c1169b4621838e302888e31047" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:46.696347 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8f66b4c1169b4621838e302888e31047" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:46.696588 2705 raft_consensus.cc:2468] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:46.697147 2843 leader_election.cc:304] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:46.697432 3040 raft_consensus.cc:2804] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:46.697518 3040 raft_consensus.cc:493] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:46.696606 2807 raft_consensus.cc:2468] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:46.697587 3040 raft_consensus.cc:3060] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:46.699626 3040 raft_consensus.cc:515] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:46.700167 3040 leader_election.cc:290] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:46.700634 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8f66b4c1169b4621838e302888e31047" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:46.700634 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8f66b4c1169b4621838e302888e31047" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:46.700932 2807 raft_consensus.cc:3060] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:46.700909 2705 raft_consensus.cc:3060] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:46.703945 2807 raft_consensus.cc:2468] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:46.703945 2705 raft_consensus.cc:2468] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:46.704466 2843 leader_election.cc:304] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:46.704867 3040 raft_consensus.cc:2804] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:46.705293 3040 raft_consensus.cc:697] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:52:46.705591 3040 consensus_queue.cc:237] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:46.707988 2586 catalog_manager.cc:5654] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:46.726094 2886 raft_consensus.cc:1275] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:46.726274 2807 raft_consensus.cc:1275] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:46.726675 3042 consensus_queue.cc:1048] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:46.726864 2954 consensus_queue.cc:1048] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:46.733659 2806 raft_consensus.cc:1275] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:46.734479 3048 consensus_queue.cc:1048] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:46.735272 2705 raft_consensus.cc:1275] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:46.735929 3048 consensus_queue.cc:1048] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:52:46.988402 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 188ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:47.253121 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 444ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:47.508683 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 709ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:47.745226 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.3:33005, user_credentials={real_user=slave}} blocked reactor thread for 1.00115e+06us
I20251212 08:52:47.746371 2586 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.4"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:47.747522 2586 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.4 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:47.760353 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 5a5ad45938984dbfbb2349fa3c85fccb (DEFAULT_TABLE table=table.4 [id=4e0eb4d6bf0e4565ac44a7b724fc04fe]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:47.760862 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a5ad45938984dbfbb2349fa3c85fccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:47.761171 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 5a5ad45938984dbfbb2349fa3c85fccb (DEFAULT_TABLE table=table.4 [id=4e0eb4d6bf0e4565ac44a7b724fc04fe]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:47.761560 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a5ad45938984dbfbb2349fa3c85fccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:47.763748 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 5a5ad45938984dbfbb2349fa3c85fccb (DEFAULT_TABLE table=table.4 [id=4e0eb4d6bf0e4565ac44a7b724fc04fe]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:47.764122 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5a5ad45938984dbfbb2349fa3c85fccb. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:47.768848 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 6b147d7c9a3c44808ebae11ec97725e0 (DEFAULT_TABLE table=table.4 [id=4e0eb4d6bf0e4565ac44a7b724fc04fe]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:47.769387 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b147d7c9a3c44808ebae11ec97725e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:47.773656 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 6b147d7c9a3c44808ebae11ec97725e0 (DEFAULT_TABLE table=table.4 [id=4e0eb4d6bf0e4565ac44a7b724fc04fe]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:47.774231 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b147d7c9a3c44808ebae11ec97725e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:47.780541 3073 tablet_bootstrap.cc:492] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:47.781778 3074 tablet_bootstrap.cc:492] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:47.781849 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 6b147d7c9a3c44808ebae11ec97725e0 (DEFAULT_TABLE table=table.4 [id=4e0eb4d6bf0e4565ac44a7b724fc04fe]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:47.782330 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b147d7c9a3c44808ebae11ec97725e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:47.782436 3073 tablet_bootstrap.cc:654] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:47.785904 3075 tablet_bootstrap.cc:492] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:47.787487 3074 tablet_bootstrap.cc:654] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:47.788072 3075 tablet_bootstrap.cc:654] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:47.798792 3075 tablet_bootstrap.cc:492] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:47.798942 3073 tablet_bootstrap.cc:492] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:47.798975 3075 ts_tablet_manager.cc:1403] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20251212 08:52:47.799034 3074 tablet_bootstrap.cc:492] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:47.799114 3073 ts_tablet_manager.cc:1403] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.010s
I20251212 08:52:47.799204 3074 ts_tablet_manager.cc:1403] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.011s
I20251212 08:52:47.799693 3075 raft_consensus.cc:359] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:47.799882 3075 raft_consensus.cc:385] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:47.799866 3074 raft_consensus.cc:359] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:47.799888 3073 raft_consensus.cc:359] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:47.800036 3075 raft_consensus.cc:740] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:47.800128 3073 raft_consensus.cc:385] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:47.800127 3074 raft_consensus.cc:385] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:47.800271 3073 raft_consensus.cc:740] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:47.800365 3074 raft_consensus.cc:740] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:47.800361 3075 consensus_queue.cc:260] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:47.800566 3073 consensus_queue.cc:260] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:47.800675 3074 consensus_queue.cc:260] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:47.801023 3075 ts_tablet_manager.cc:1434] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:47.801148 3073 ts_tablet_manager.cc:1434] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:52:47.801352 3075 tablet_bootstrap.cc:492] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:47.801438 3073 tablet_bootstrap.cc:492] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:47.802137 3074 ts_tablet_manager.cc:1434] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251212 08:52:47.802487 3074 tablet_bootstrap.cc:492] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:47.803431 3073 tablet_bootstrap.cc:654] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:47.803471 3075 tablet_bootstrap.cc:654] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:47.804307 3074 tablet_bootstrap.cc:654] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:47.812866 3067 raft_consensus.cc:493] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:47.813094 3067 raft_consensus.cc:515] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:47.814044 3067 leader_election.cc:290] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:47.815124 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b147d7c9a3c44808ebae11ec97725e0" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
W20251212 08:52:47.816241 2667 leader_election.cc:343] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879): Illegal state: must be running to vote when last-logged opid is not known
I20251212 08:52:47.816737 3074 tablet_bootstrap.cc:492] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:47.817023 3074 ts_tablet_manager.cc:1403] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.015s user 0.008s sys 0.002s
I20251212 08:52:47.816965 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b147d7c9a3c44808ebae11ec97725e0" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:47.818081 3074 raft_consensus.cc:359] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:47.818480 3074 raft_consensus.cc:385] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:47.818636 3074 raft_consensus.cc:740] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
W20251212 08:52:47.818919 2667 leader_election.cc:343] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005): Illegal state: must be running to vote when last-logged opid is not known
I20251212 08:52:47.819161 2667 leader_election.cc:304] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cab108f9abb4e1c973338419932d96c; no voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78
I20251212 08:52:47.820073 3067 raft_consensus.cc:2749] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251212 08:52:47.820417 3074 consensus_queue.cc:260] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:47.822389 3074 ts_tablet_manager.cc:1434] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251212 08:52:47.828140 3075 tablet_bootstrap.cc:492] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:47.828213 3073 tablet_bootstrap.cc:492] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:47.828509 3075 ts_tablet_manager.cc:1403] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.027s user 0.021s sys 0.006s
I20251212 08:52:47.828558 3073 ts_tablet_manager.cc:1403] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.027s user 0.016s sys 0.010s
I20251212 08:52:47.829236 3073 raft_consensus.cc:359] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:47.829452 3075 raft_consensus.cc:359] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:47.829583 3073 raft_consensus.cc:385] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:47.829785 3073 raft_consensus.cc:740] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:47.829816 3075 raft_consensus.cc:385] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:47.829980 3075 raft_consensus.cc:740] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:47.830076 3073 consensus_queue.cc:260] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:47.830255 3075 consensus_queue.cc:260] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:47.830794 3073 ts_tablet_manager.cc:1434] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:52:47.831569 3075 ts_tablet_manager.cc:1434] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:52:47.878736 3068 raft_consensus.cc:493] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:47.878928 3068 raft_consensus.cc:515] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:47.879747 3068 leader_election.cc:290] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:47.880201 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5a5ad45938984dbfbb2349fa3c85fccb" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:47.880245 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5a5ad45938984dbfbb2349fa3c85fccb" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:47.880496 2886 raft_consensus.cc:2468] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:47.880518 2705 raft_consensus.cc:2468] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:47.880911 2765 leader_election.cc:304] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:47.881170 3068 raft_consensus.cc:2804] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:47.881279 3068 raft_consensus.cc:493] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:47.881349 3068 raft_consensus.cc:3060] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:47.884038 3068 raft_consensus.cc:515] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:47.884920 3068 leader_election.cc:290] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:47.885296 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5a5ad45938984dbfbb2349fa3c85fccb" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:47.885303 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "5a5ad45938984dbfbb2349fa3c85fccb" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:47.885540 2886 raft_consensus.cc:3060] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:47.885630 2705 raft_consensus.cc:3060] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:47.887925 2886 raft_consensus.cc:2468] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:47.888371 2765 leader_election.cc:304] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:47.888670 3082 raft_consensus.cc:2804] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:47.888823 3082 raft_consensus.cc:697] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:47.889140 3082 consensus_queue.cc:237] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:47.890542 2705 raft_consensus.cc:2468] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:47.891702 2587 catalog_manager.cc:5654] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:48.074903 3040 raft_consensus.cc:493] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:48.075243 3040 raft_consensus.cc:515] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:48.076047 3040 leader_election.cc:290] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:52:48.076566 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b147d7c9a3c44808ebae11ec97725e0" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:48.076803 2705 raft_consensus.cc:2468] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:48.077265 2843 leader_election.cc:304] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:48.077539 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b147d7c9a3c44808ebae11ec97725e0" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:48.077638 3040 raft_consensus.cc:2804] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:48.077782 3040 raft_consensus.cc:493] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:48.077915 3040 raft_consensus.cc:3060] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:48.077757 2807 raft_consensus.cc:2468] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:48.080128 3040 raft_consensus.cc:515] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:48.080766 3040 leader_election.cc:290] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:52:48.081208 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b147d7c9a3c44808ebae11ec97725e0" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:48.081226 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6b147d7c9a3c44808ebae11ec97725e0" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:48.081423 2807 raft_consensus.cc:3060] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:48.081449 2705 raft_consensus.cc:3060] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:48.083582 2705 raft_consensus.cc:2468] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:48.083604 2807 raft_consensus.cc:2468] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:48.084012 2845 leader_election.cc:304] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:48.084334 3040 raft_consensus.cc:2804] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:48.084707 3040 raft_consensus.cc:697] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:52:48.085009 3040 consensus_queue.cc:237] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:48.087988 2587 catalog_manager.cc:5654] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:48.169114 2807 raft_consensus.cc:1275] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:48.169301 2705 raft_consensus.cc:1275] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:48.169577 2886 raft_consensus.cc:1275] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:48.169641 2704 raft_consensus.cc:1275] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:48.169875 3084 consensus_queue.cc:1048] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:48.170066 3082 consensus_queue.cc:1048] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:48.170233 3040 consensus_queue.cc:1048] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:48.170537 3082 consensus_queue.cc:1048] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:52:48.428555 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 187ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:48.688313 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 441ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:48.967365 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 701ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:49.195114 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.3:33005, user_credentials={real_user=slave}} blocked reactor thread for 1.00798e+06us
I20251212 08:52:49.196972 2590 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.5"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:49.198020 2590 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.5 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:49.220402 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 8638463f714e4dddbf5cec9d89890e4e (DEFAULT_TABLE table=table.5 [id=91128a33e48840c5b85da99057eed613]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:49.220868 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 8638463f714e4dddbf5cec9d89890e4e (DEFAULT_TABLE table=table.5 [id=91128a33e48840c5b85da99057eed613]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:49.220997 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8638463f714e4dddbf5cec9d89890e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:49.221236 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8638463f714e4dddbf5cec9d89890e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:49.221396 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 62b27193ad064f6e87f51a3e3a8756bc (DEFAULT_TABLE table=table.5 [id=91128a33e48840c5b85da99057eed613]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:49.221717 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62b27193ad064f6e87f51a3e3a8756bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:49.220454 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 8638463f714e4dddbf5cec9d89890e4e (DEFAULT_TABLE table=table.5 [id=91128a33e48840c5b85da99057eed613]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:49.224617 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8638463f714e4dddbf5cec9d89890e4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:49.229060 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 62b27193ad064f6e87f51a3e3a8756bc (DEFAULT_TABLE table=table.5 [id=91128a33e48840c5b85da99057eed613]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:49.229931 3105 tablet_bootstrap.cc:492] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:49.230544 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62b27193ad064f6e87f51a3e3a8756bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:49.230475 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 62b27193ad064f6e87f51a3e3a8756bc (DEFAULT_TABLE table=table.5 [id=91128a33e48840c5b85da99057eed613]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:49.230881 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 62b27193ad064f6e87f51a3e3a8756bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:49.232124 3105 tablet_bootstrap.cc:654] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:49.246934 3105 tablet_bootstrap.cc:492] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:49.247853 3105 ts_tablet_manager.cc:1403] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.000s
I20251212 08:52:49.249338 3105 raft_consensus.cc:359] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:49.251195 3105 raft_consensus.cc:385] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:49.251356 3105 raft_consensus.cc:740] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:49.251714 3105 consensus_queue.cc:260] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:49.250751 3107 tablet_bootstrap.cc:492] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:49.254528 3107 tablet_bootstrap.cc:654] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:49.256175 3105 ts_tablet_manager.cc:1434] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20251212 08:52:49.256508 3108 tablet_bootstrap.cc:492] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:49.257584 3105 tablet_bootstrap.cc:492] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:49.260329 3108 tablet_bootstrap.cc:654] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:49.261129 3105 tablet_bootstrap.cc:654] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:49.272123 3108 tablet_bootstrap.cc:492] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:49.272413 3108 ts_tablet_manager.cc:1403] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.005s
I20251212 08:52:49.274698 3107 tablet_bootstrap.cc:492] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:49.274959 3107 ts_tablet_manager.cc:1403] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.024s user 0.006s sys 0.006s
I20251212 08:52:49.275336 3105 tablet_bootstrap.cc:492] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:49.275353 3108 raft_consensus.cc:359] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:49.275566 3107 raft_consensus.cc:359] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:49.275574 3105 ts_tablet_manager.cc:1403] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.012s
I20251212 08:52:49.275715 3107 raft_consensus.cc:385] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:49.275923 3107 raft_consensus.cc:740] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:49.275738 3108 raft_consensus.cc:385] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:49.277022 3108 raft_consensus.cc:740] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:49.276559 3105 raft_consensus.cc:359] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:49.277410 3105 raft_consensus.cc:385] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:49.277365 3108 consensus_queue.cc:260] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:49.277526 3105 raft_consensus.cc:740] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:49.277915 3105 consensus_queue.cc:260] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:49.279877 3105 ts_tablet_manager.cc:1434] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:52:49.280793 3108 ts_tablet_manager.cc:1434] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251212 08:52:49.281942 3108 tablet_bootstrap.cc:492] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:49.284030 3108 tablet_bootstrap.cc:654] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:49.282161 3107 consensus_queue.cc:260] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:49.285734 3107 ts_tablet_manager.cc:1434] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.011s user 0.001s sys 0.001s
I20251212 08:52:49.286168 3107 tablet_bootstrap.cc:492] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:49.287961 3107 tablet_bootstrap.cc:654] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:49.291628 3107 tablet_bootstrap.cc:492] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:49.291882 3107 ts_tablet_manager.cc:1403] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20251212 08:52:49.292901 3107 raft_consensus.cc:359] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:49.293183 3107 raft_consensus.cc:385] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:49.293287 3107 raft_consensus.cc:740] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:49.293603 3107 consensus_queue.cc:260] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:49.295765 3107 ts_tablet_manager.cc:1434] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:52:49.298518 3108 tablet_bootstrap.cc:492] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:49.298769 3108 ts_tablet_manager.cc:1403] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.017s user 0.001s sys 0.004s
I20251212 08:52:49.299537 3108 raft_consensus.cc:359] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:49.299785 3108 raft_consensus.cc:385] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:49.299921 3108 raft_consensus.cc:740] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:49.300256 3108 consensus_queue.cc:260] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:49.300945 3108 ts_tablet_manager.cc:1434] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:52:49.323649 3100 raft_consensus.cc:493] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:49.323871 3100 raft_consensus.cc:515] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:49.324649 3100 leader_election.cc:290] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:52:49.325307 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "62b27193ad064f6e87f51a3e3a8756bc" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:49.325421 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "62b27193ad064f6e87f51a3e3a8756bc" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:49.325562 2705 raft_consensus.cc:2468] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:49.325558 2807 raft_consensus.cc:2468] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:49.326022 2845 leader_election.cc:304] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:49.326365 3100 raft_consensus.cc:2804] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:49.326486 3100 raft_consensus.cc:493] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:49.326690 3100 raft_consensus.cc:3060] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:49.329408 3100 raft_consensus.cc:515] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:49.330641 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "62b27193ad064f6e87f51a3e3a8756bc" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:49.330696 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "62b27193ad064f6e87f51a3e3a8756bc" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:49.330859 2807 raft_consensus.cc:3060] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:49.330907 2705 raft_consensus.cc:3060] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:49.333290 3100 leader_election.cc:290] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:52:49.333945 2807 raft_consensus.cc:2468] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:49.334532 2845 leader_election.cc:304] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:49.335000 3100 raft_consensus.cc:2804] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:49.336905 2705 raft_consensus.cc:2468] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:49.346480 3100 raft_consensus.cc:697] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:52:49.346992 3100 consensus_queue.cc:237] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:49.355310 2587 catalog_manager.cc:5654] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:49.417469 3068 raft_consensus.cc:493] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:49.417672 3068 raft_consensus.cc:515] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:49.418386 3068 leader_election.cc:290] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:49.418752 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8638463f714e4dddbf5cec9d89890e4e" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:49.418805 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8638463f714e4dddbf5cec9d89890e4e" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:49.419001 2887 raft_consensus.cc:2468] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:49.419081 2705 raft_consensus.cc:2468] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:49.419433 2763 leader_election.cc:304] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:49.419688 3068 raft_consensus.cc:2804] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:49.419824 3068 raft_consensus.cc:493] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:49.419926 3068 raft_consensus.cc:3060] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:49.422151 3068 raft_consensus.cc:515] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:49.422719 3068 leader_election.cc:290] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:49.423118 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8638463f714e4dddbf5cec9d89890e4e" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:49.423187 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8638463f714e4dddbf5cec9d89890e4e" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:49.423312 2887 raft_consensus.cc:3060] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:49.423425 2705 raft_consensus.cc:3060] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:49.425964 2887 raft_consensus.cc:2468] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:49.426266 2705 raft_consensus.cc:2468] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:49.426335 2765 leader_election.cc:304] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:49.426705 3068 raft_consensus.cc:2804] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:49.427084 3068 raft_consensus.cc:697] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:49.427357 3068 consensus_queue.cc:237] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:49.429955 2587 catalog_manager.cc:5654] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:49.470189 2807 raft_consensus.cc:1275] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:49.470523 2705 raft_consensus.cc:1275] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:49.470858 3114 consensus_queue.cc:1048] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:49.471480 3100 consensus_queue.cc:1048] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:49.471637 2887 raft_consensus.cc:1275] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:49.472430 3117 consensus_queue.cc:1048] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:49.473016 2705 raft_consensus.cc:1275] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:49.476568 3068 consensus_queue.cc:1048] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:49.493927 3120 mvcc.cc:204] Tried to move back new op lower bound from 7231609116545224704 to 7231609116049211392. Current Snapshot: MvccSnapshot[applied={T|T < 7231609116545224704}]
W20251212 08:52:49.751083 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 200ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:50.008852 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 457ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:50.267105 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 715ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:50.495553 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.3:33005, user_credentials={real_user=slave}} blocked reactor thread for 1.00115e+06us
I20251212 08:52:50.497263 2588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.6"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:50.498348 2588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.6 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:50.509696 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 2fd487ada3bd4c41b299172d81e63b7e (DEFAULT_TABLE table=table.6 [id=6af1f03cb45a41fba9238baa6240383c]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:50.510183 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd487ada3bd4c41b299172d81e63b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:50.510681 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 2fd487ada3bd4c41b299172d81e63b7e (DEFAULT_TABLE table=table.6 [id=6af1f03cb45a41fba9238baa6240383c]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:50.510782 2695 tablet_service.cc:1505] Processing CreateTablet for tablet c747caeb72d148f1a70b0ce32ea9f7ff (DEFAULT_TABLE table=table.6 [id=6af1f03cb45a41fba9238baa6240383c]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:50.511078 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd487ada3bd4c41b299172d81e63b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:50.511291 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c747caeb72d148f1a70b0ce32ea9f7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:50.516769 2795 tablet_service.cc:1505] Processing CreateTablet for tablet c747caeb72d148f1a70b0ce32ea9f7ff (DEFAULT_TABLE table=table.6 [id=6af1f03cb45a41fba9238baa6240383c]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:50.517037 2873 tablet_service.cc:1505] Processing CreateTablet for tablet c747caeb72d148f1a70b0ce32ea9f7ff (DEFAULT_TABLE table=table.6 [id=6af1f03cb45a41fba9238baa6240383c]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:50.517198 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c747caeb72d148f1a70b0ce32ea9f7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:50.517344 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c747caeb72d148f1a70b0ce32ea9f7ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:50.516769 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 2fd487ada3bd4c41b299172d81e63b7e (DEFAULT_TABLE table=table.6 [id=6af1f03cb45a41fba9238baa6240383c]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:50.520800 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2fd487ada3bd4c41b299172d81e63b7e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:50.532999 3140 tablet_bootstrap.cc:492] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:50.535702 3140 tablet_bootstrap.cc:654] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:50.540808 3139 tablet_bootstrap.cc:492] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:50.542091 3139 tablet_bootstrap.cc:654] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:50.542666 3142 tablet_bootstrap.cc:492] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:50.543848 3140 tablet_bootstrap.cc:492] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:50.543951 3139 tablet_bootstrap.cc:492] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:50.543975 3140 ts_tablet_manager.cc:1403] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20251212 08:52:50.544070 3139 ts_tablet_manager.cc:1403] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:52:50.544392 3142 tablet_bootstrap.cc:654] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:50.544740 3140 raft_consensus.cc:359] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:50.544893 3140 raft_consensus.cc:385] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:50.544940 3140 raft_consensus.cc:740] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:50.545135 3140 consensus_queue.cc:260] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:50.545442 3139 raft_consensus.cc:359] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:50.545624 3139 raft_consensus.cc:385] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:50.545680 3139 raft_consensus.cc:740] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:50.545681 3140 ts_tablet_manager.cc:1434] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:50.545939 3140 tablet_bootstrap.cc:492] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:50.545899 3139 consensus_queue.cc:260] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:50.547417 3140 tablet_bootstrap.cc:654] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:50.547538 3139 ts_tablet_manager.cc:1434] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:52:50.547690 3142 tablet_bootstrap.cc:492] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:50.547838 3142 ts_tablet_manager.cc:1403] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251212 08:52:50.547955 3139 tablet_bootstrap.cc:492] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:50.549863 3139 tablet_bootstrap.cc:654] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:50.549800 3142 raft_consensus.cc:359] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:50.550010 3142 raft_consensus.cc:385] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:50.550247 3142 raft_consensus.cc:740] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:50.551147 3142 consensus_queue.cc:260] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:50.551791 3142 ts_tablet_manager.cc:1434] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251212 08:52:50.552175 3142 tablet_bootstrap.cc:492] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:50.555828 3139 tablet_bootstrap.cc:492] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:50.555962 3139 ts_tablet_manager.cc:1403] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20251212 08:52:50.556334 3142 tablet_bootstrap.cc:654] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:50.556771 3139 raft_consensus.cc:359] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:50.556953 3139 raft_consensus.cc:385] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:50.557021 3139 raft_consensus.cc:740] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:50.557253 3139 consensus_queue.cc:260] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:50.557992 3139 ts_tablet_manager.cc:1434] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:52:50.560220 3140 tablet_bootstrap.cc:492] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:50.560377 3140 ts_tablet_manager.cc:1403] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.007s
I20251212 08:52:50.561025 3140 raft_consensus.cc:359] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:50.561151 3140 raft_consensus.cc:385] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:50.561206 3140 raft_consensus.cc:740] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:50.561223 3142 tablet_bootstrap.cc:492] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:50.561404 3142 ts_tablet_manager.cc:1403] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20251212 08:52:50.561905 3142 raft_consensus.cc:359] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:50.562032 3142 raft_consensus.cc:385] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:50.562073 3142 raft_consensus.cc:740] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:50.562201 3142 consensus_queue.cc:260] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:50.562547 3142 ts_tablet_manager.cc:1434] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251212 08:52:50.561412 3140 consensus_queue.cc:260] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:50.563052 3140 ts_tablet_manager.cc:1434] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:52:50.836217 3088 raft_consensus.cc:493] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:50.836438 3088 raft_consensus.cc:515] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:50.837075 3088 leader_election.cc:290] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:50.837535 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c747caeb72d148f1a70b0ce32ea9f7ff" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:50.837565 2806 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c747caeb72d148f1a70b0ce32ea9f7ff" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:50.837908 2806 raft_consensus.cc:2468] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:50.837774 2887 raft_consensus.cc:2468] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:50.838376 2667 leader_election.cc:304] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:50.838603 3088 raft_consensus.cc:2804] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:50.838677 3088 raft_consensus.cc:493] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:50.838737 3088 raft_consensus.cc:3060] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:50.840919 3088 raft_consensus.cc:515] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:50.841879 2806 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c747caeb72d148f1a70b0ce32ea9f7ff" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:50.842041 2806 raft_consensus.cc:3060] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:50.843562 3088 leader_election.cc:290] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:50.844640 2806 raft_consensus.cc:2468] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:50.844909 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c747caeb72d148f1a70b0ce32ea9f7ff" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:50.845110 2887 raft_consensus.cc:3060] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:50.848441 2887 raft_consensus.cc:2468] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:50.848582 2667 leader_election.cc:304] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:50.848876 3088 raft_consensus.cc:2804] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:50.849486 3088 raft_consensus.cc:697] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:52:50.849864 3088 consensus_queue.cc:237] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:50.856941 2588 catalog_manager.cc:5654] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:50.926393 3134 raft_consensus.cc:493] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:50.926585 3134 raft_consensus.cc:515] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:50.927294 3134 leader_election.cc:290] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:50.927788 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fd487ada3bd4c41b299172d81e63b7e" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:50.927919 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fd487ada3bd4c41b299172d81e63b7e" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:50.927987 2887 raft_consensus.cc:2468] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:50.928136 2705 raft_consensus.cc:2468] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:50.928515 2765 leader_election.cc:304] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:50.928871 3134 raft_consensus.cc:2804] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:50.928958 3134 raft_consensus.cc:493] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:50.929039 3134 raft_consensus.cc:3060] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:50.931308 3134 raft_consensus.cc:515] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:50.931856 3134 leader_election.cc:290] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:50.932212 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fd487ada3bd4c41b299172d81e63b7e" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:50.932211 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "2fd487ada3bd4c41b299172d81e63b7e" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:50.932449 2705 raft_consensus.cc:3060] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:50.932449 2887 raft_consensus.cc:3060] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:50.935456 2887 raft_consensus.cc:2468] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:50.935456 2705 raft_consensus.cc:2468] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:50.935945 2765 leader_election.cc:304] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:50.936230 3134 raft_consensus.cc:2804] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:50.936647 3134 raft_consensus.cc:697] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:50.936892 3134 consensus_queue.cc:237] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:50.939888 2590 catalog_manager.cc:5654] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:50.983405 2887 raft_consensus.cc:1275] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:50.983484 2806 raft_consensus.cc:1275] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:50.983928 3088 consensus_queue.cc:1048] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:50.984220 3148 consensus_queue.cc:1048] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:50.984784 2887 raft_consensus.cc:1275] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:50.984946 2705 raft_consensus.cc:1275] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:50.985349 3134 consensus_queue.cc:1048] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:50.987455 3134 consensus_queue.cc:1048] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:52:51.195617 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 141ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:51.451833 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 398ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:51.709769 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 655ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:51.965489 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 913ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:51.998521 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00114e+06us
I20251212 08:52:51.999567 2589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.7"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:52.000218 2589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.7 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:52.008597 2793 tablet_service.cc:1505] Processing CreateTablet for tablet ff97ced42b1543b2a2b41d4106de4e5c (DEFAULT_TABLE table=table.7 [id=1dfdcf48b43b4a8690c4c2aadf08cc9f]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:52.009131 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff97ced42b1543b2a2b41d4106de4e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:52.009152 2873 tablet_service.cc:1505] Processing CreateTablet for tablet ff97ced42b1543b2a2b41d4106de4e5c (DEFAULT_TABLE table=table.7 [id=1dfdcf48b43b4a8690c4c2aadf08cc9f]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:52.010349 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff97ced42b1543b2a2b41d4106de4e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:52.009938 2694 tablet_service.cc:1505] Processing CreateTablet for tablet ff97ced42b1543b2a2b41d4106de4e5c (DEFAULT_TABLE table=table.7 [id=1dfdcf48b43b4a8690c4c2aadf08cc9f]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:52.011669 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff97ced42b1543b2a2b41d4106de4e5c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:52.013248 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 31333c1e9c574d0bb2f38ca25d972d70 (DEFAULT_TABLE table=table.7 [id=1dfdcf48b43b4a8690c4c2aadf08cc9f]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:52.013662 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31333c1e9c574d0bb2f38ca25d972d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:52.022172 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 31333c1e9c574d0bb2f38ca25d972d70 (DEFAULT_TABLE table=table.7 [id=1dfdcf48b43b4a8690c4c2aadf08cc9f]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:52.022116 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 31333c1e9c574d0bb2f38ca25d972d70 (DEFAULT_TABLE table=table.7 [id=1dfdcf48b43b4a8690c4c2aadf08cc9f]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:52.022611 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31333c1e9c574d0bb2f38ca25d972d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:52.022737 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31333c1e9c574d0bb2f38ca25d972d70. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:52.022812 3171 tablet_bootstrap.cc:492] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:52.027334 3170 tablet_bootstrap.cc:492] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:52.030256 3170 tablet_bootstrap.cc:654] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:52.030944 3171 tablet_bootstrap.cc:654] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:52.035907 3169 tablet_bootstrap.cc:492] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:52.037595 3169 tablet_bootstrap.cc:654] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:52.041726 3171 tablet_bootstrap.cc:492] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:52.041929 3171 ts_tablet_manager.cc:1403] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.000s
I20251212 08:52:52.042905 3171 raft_consensus.cc:359] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:52.043457 3170 tablet_bootstrap.cc:492] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:52.043936 3171 raft_consensus.cc:385] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:52.044129 3170 ts_tablet_manager.cc:1403] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.007s
I20251212 08:52:52.044137 3171 raft_consensus.cc:740] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:52.044492 3171 consensus_queue.cc:260] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:52.044837 3169 tablet_bootstrap.cc:492] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:52.045012 3169 ts_tablet_manager.cc:1403] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20251212 08:52:52.045406 3170 raft_consensus.cc:359] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:52.045661 3170 raft_consensus.cc:385] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:52.045857 3170 raft_consensus.cc:740] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:52.045951 3169 raft_consensus.cc:359] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:52.046139 3169 raft_consensus.cc:385] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:52.046164 3170 consensus_queue.cc:260] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:52.046273 3169 raft_consensus.cc:740] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:52.046454 3171 ts_tablet_manager.cc:1434] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20251212 08:52:52.046530 3169 consensus_queue.cc:260] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:52.046763 3170 ts_tablet_manager.cc:1434] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:52.046770 3171 tablet_bootstrap.cc:492] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:52.047173 3169 ts_tablet_manager.cc:1434] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251212 08:52:52.048832 3169 tablet_bootstrap.cc:492] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:52.047063 3170 tablet_bootstrap.cc:492] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:52.051108 3169 tablet_bootstrap.cc:654] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:52.052693 3170 tablet_bootstrap.cc:654] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:52.054347 3171 tablet_bootstrap.cc:654] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:52.055794 3170 tablet_bootstrap.cc:492] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:52.055955 3170 ts_tablet_manager.cc:1403] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20251212 08:52:52.056532 3170 raft_consensus.cc:359] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:52.056710 3170 raft_consensus.cc:385] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:52.056795 3170 raft_consensus.cc:740] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:52.057006 3170 consensus_queue.cc:260] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:52.057303 3169 tablet_bootstrap.cc:492] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:52.057507 3170 ts_tablet_manager.cc:1434] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251212 08:52:52.057505 3169 ts_tablet_manager.cc:1403] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20251212 08:52:52.058064 3171 tablet_bootstrap.cc:492] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:52.058228 3171 ts_tablet_manager.cc:1403] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.002s
I20251212 08:52:52.058287 3169 raft_consensus.cc:359] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:52.058494 3169 raft_consensus.cc:385] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:52.058593 3169 raft_consensus.cc:740] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:52.058843 3169 consensus_queue.cc:260] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:52.058898 3171 raft_consensus.cc:359] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:52.059238 3171 raft_consensus.cc:385] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:52.059386 3171 raft_consensus.cc:740] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:52.059701 3171 consensus_queue.cc:260] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:52.060088 3169 ts_tablet_manager.cc:1434] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:52:52.061178 3171 ts_tablet_manager.cc:1434] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251212 08:52:52.152873 3134 raft_consensus.cc:493] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:52.153069 3134 raft_consensus.cc:515] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:52.153556 3134 leader_election.cc:290] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:52.153982 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff97ced42b1543b2a2b41d4106de4e5c" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:52.153960 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff97ced42b1543b2a2b41d4106de4e5c" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:52.154175 2705 raft_consensus.cc:2468] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:52.154198 2887 raft_consensus.cc:2468] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:52.154584 2763 leader_election.cc:304] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:52.154901 3134 raft_consensus.cc:2804] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:52.154999 3134 raft_consensus.cc:493] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:52.155082 3134 raft_consensus.cc:3060] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:52.157027 3134 raft_consensus.cc:515] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:52.157660 3134 leader_election.cc:290] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:52.158047 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff97ced42b1543b2a2b41d4106de4e5c" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:52.158147 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ff97ced42b1543b2a2b41d4106de4e5c" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:52.158226 2887 raft_consensus.cc:3060] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:52.158344 2705 raft_consensus.cc:3060] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:52.161008 2887 raft_consensus.cc:2468] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:52.161047 2705 raft_consensus.cc:2468] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:52.161581 2763 leader_election.cc:304] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:52.161885 3134 raft_consensus.cc:2804] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:52.162436 3134 raft_consensus.cc:697] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:52.162743 3134 consensus_queue.cc:237] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:52.166884 2587 catalog_manager.cc:5654] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:52.191840 3100 raft_consensus.cc:493] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:52.192072 3100 raft_consensus.cc:515] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:52.192832 3100 leader_election.cc:290] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:52:52.193125 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "31333c1e9c574d0bb2f38ca25d972d70" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:52.193120 2806 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "31333c1e9c574d0bb2f38ca25d972d70" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:52.193493 2705 raft_consensus.cc:2468] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:52.193522 2806 raft_consensus.cc:2468] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:52.193895 2843 leader_election.cc:304] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:52.194157 3100 raft_consensus.cc:2804] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:52.194272 3100 raft_consensus.cc:493] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:52.194362 3100 raft_consensus.cc:3060] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:52.196339 3100 raft_consensus.cc:515] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:52.197013 3100 leader_election.cc:290] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:52:52.197396 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "31333c1e9c574d0bb2f38ca25d972d70" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:52.197441 2806 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "31333c1e9c574d0bb2f38ca25d972d70" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:52.197633 2705 raft_consensus.cc:3060] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:52.197654 2806 raft_consensus.cc:3060] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:52.200927 2806 raft_consensus.cc:2468] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:52.200927 2705 raft_consensus.cc:2468] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:52.201479 2843 leader_election.cc:304] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:52.201824 3100 raft_consensus.cc:2804] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:52.202226 3100 raft_consensus.cc:697] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:52:52.202526 3100 consensus_queue.cc:237] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:52.205971 2585 catalog_manager.cc:5654] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:52.219735 2806 raft_consensus.cc:1275] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:52.220652 2705 raft_consensus.cc:1275] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:52.220654 3100 consensus_queue.cc:1048] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:52.221757 2705 raft_consensus.cc:1275] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:52.221952 2887 raft_consensus.cc:1275] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:52.222327 3180 consensus_queue.cc:1048] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:52.222267 3184 consensus_queue.cc:1048] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:52.223011 3184 consensus_queue.cc:1048] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:52:52.426694 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 116ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:52.686558 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 376ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:52.948084 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 636ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:53.208622 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 898ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:53.251487 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.0012e+06us
I20251212 08:52:53.253122 2585 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.8"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:53.253955 2585 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.8 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:53.262998 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 85bd9f82cab946bfa260338f237a589d (DEFAULT_TABLE table=table.8 [id=31bb6f04d37d4feebf325f81a6fd0a3b]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:53.263320 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 85bd9f82cab946bfa260338f237a589d (DEFAULT_TABLE table=table.8 [id=31bb6f04d37d4feebf325f81a6fd0a3b]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:53.263538 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85bd9f82cab946bfa260338f237a589d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:53.263655 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85bd9f82cab946bfa260338f237a589d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:53.266551 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 85bd9f82cab946bfa260338f237a589d (DEFAULT_TABLE table=table.8 [id=31bb6f04d37d4feebf325f81a6fd0a3b]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:53.266561 2695 tablet_service.cc:1505] Processing CreateTablet for tablet d897186bf4c3441caefa0c2a5d83fc4c (DEFAULT_TABLE table=table.8 [id=31bb6f04d37d4feebf325f81a6fd0a3b]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:53.266970 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85bd9f82cab946bfa260338f237a589d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:53.267154 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d897186bf4c3441caefa0c2a5d83fc4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:53.270573 2793 tablet_service.cc:1505] Processing CreateTablet for tablet d897186bf4c3441caefa0c2a5d83fc4c (DEFAULT_TABLE table=table.8 [id=31bb6f04d37d4feebf325f81a6fd0a3b]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:53.271158 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d897186bf4c3441caefa0c2a5d83fc4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:53.271552 2876 tablet_service.cc:1505] Processing CreateTablet for tablet d897186bf4c3441caefa0c2a5d83fc4c (DEFAULT_TABLE table=table.8 [id=31bb6f04d37d4feebf325f81a6fd0a3b]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:53.271901 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d897186bf4c3441caefa0c2a5d83fc4c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:53.286387 3204 tablet_bootstrap.cc:492] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:53.286584 3205 tablet_bootstrap.cc:492] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:53.288378 3204 tablet_bootstrap.cc:654] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:53.289549 3205 tablet_bootstrap.cc:654] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:53.293648 3204 tablet_bootstrap.cc:492] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:53.293802 3204 ts_tablet_manager.cc:1403] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20251212 08:52:53.294384 3205 tablet_bootstrap.cc:492] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:53.294498 3205 ts_tablet_manager.cc:1403] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20251212 08:52:53.294637 3204 raft_consensus.cc:359] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:53.294798 3204 raft_consensus.cc:385] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:53.295141 3205 raft_consensus.cc:359] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:53.295302 3205 raft_consensus.cc:385] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:53.295353 3205 raft_consensus.cc:740] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:53.295579 3205 consensus_queue.cc:260] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:53.295818 3204 raft_consensus.cc:740] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:53.296116 3204 consensus_queue.cc:260] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:53.296267 3205 ts_tablet_manager.cc:1434] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:52:53.298072 3205 tablet_bootstrap.cc:492] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:53.298310 3204 ts_tablet_manager.cc:1434] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:52:53.298611 3204 tablet_bootstrap.cc:492] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:53.300590 3203 tablet_bootstrap.cc:492] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:53.301182 3204 tablet_bootstrap.cc:654] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:53.302021 3205 tablet_bootstrap.cc:654] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:53.302537 3203 tablet_bootstrap.cc:654] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:53.304499 3204 tablet_bootstrap.cc:492] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:53.304718 3204 ts_tablet_manager.cc:1403] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251212 08:52:53.304771 3205 tablet_bootstrap.cc:492] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:53.304877 3205 ts_tablet_manager.cc:1403] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20251212 08:52:53.305585 3205 raft_consensus.cc:359] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:53.305585 3204 raft_consensus.cc:359] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:53.306263 3203 tablet_bootstrap.cc:492] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:53.306447 3204 raft_consensus.cc:385] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:53.306506 3203 ts_tablet_manager.cc:1403] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20251212 08:52:53.306540 3204 raft_consensus.cc:740] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:53.306794 3204 consensus_queue.cc:260] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:53.307685 3204 ts_tablet_manager.cc:1434] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:52:53.307729 3203 raft_consensus.cc:359] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:53.307881 3203 raft_consensus.cc:385] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:53.307946 3203 raft_consensus.cc:740] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:53.308120 3203 consensus_queue.cc:260] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:53.308596 3203 ts_tablet_manager.cc:1434] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:53.308830 3203 tablet_bootstrap.cc:492] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:53.310230 3203 tablet_bootstrap.cc:654] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:53.306370 3205 raft_consensus.cc:385] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:53.311241 3205 raft_consensus.cc:740] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:53.311962 3203 tablet_bootstrap.cc:492] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:53.312242 3203 ts_tablet_manager.cc:1403] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:52:53.312491 3205 consensus_queue.cc:260] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:53.313045 3203 raft_consensus.cc:359] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:53.313364 3205 ts_tablet_manager.cc:1434] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20251212 08:52:53.313372 3203 raft_consensus.cc:385] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:53.313591 3203 raft_consensus.cc:740] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:53.313809 3203 consensus_queue.cc:260] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:53.314440 3203 ts_tablet_manager.cc:1434] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:53.347281 3148 raft_consensus.cc:493] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:53.347476 3148 raft_consensus.cc:515] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:53.348192 3148 leader_election.cc:290] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:53.348641 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d897186bf4c3441caefa0c2a5d83fc4c" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:53.348647 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d897186bf4c3441caefa0c2a5d83fc4c" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:53.348811 2887 raft_consensus.cc:2468] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:53.348915 2807 raft_consensus.cc:2468] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:53.349190 2667 leader_election.cc:304] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:53.349397 3148 raft_consensus.cc:2804] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:53.349462 3148 raft_consensus.cc:493] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:53.349509 3148 raft_consensus.cc:3060] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:53.351488 3148 raft_consensus.cc:515] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:53.352023 3148 leader_election.cc:290] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:53.352478 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d897186bf4c3441caefa0c2a5d83fc4c" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:53.352519 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d897186bf4c3441caefa0c2a5d83fc4c" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:53.352669 2807 raft_consensus.cc:3060] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:53.352669 2887 raft_consensus.cc:3060] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:53.355755 2887 raft_consensus.cc:2468] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:53.355755 2807 raft_consensus.cc:2468] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:53.356369 2667 leader_election.cc:304] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:53.356638 3148 raft_consensus.cc:2804] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:53.357417 3148 raft_consensus.cc:697] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:52:53.357756 3148 consensus_queue.cc:237] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:53.361271 2589 catalog_manager.cc:5654] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:53.418593 3184 raft_consensus.cc:493] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:53.418824 3184 raft_consensus.cc:515] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:53.419586 3184 leader_election.cc:290] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:53.419894 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85bd9f82cab946bfa260338f237a589d" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:53.419998 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85bd9f82cab946bfa260338f237a589d" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:53.420085 2887 raft_consensus.cc:2468] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:53.420238 2704 raft_consensus.cc:2468] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:53.420555 2765 leader_election.cc:304] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:53.420816 3184 raft_consensus.cc:2804] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:53.420898 3184 raft_consensus.cc:493] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:53.420974 3184 raft_consensus.cc:3060] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:53.423017 3184 raft_consensus.cc:515] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:53.423720 3184 leader_election.cc:290] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:53.423956 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85bd9f82cab946bfa260338f237a589d" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:53.424135 2887 raft_consensus.cc:3060] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:53.424108 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "85bd9f82cab946bfa260338f237a589d" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:53.424239 2704 raft_consensus.cc:3060] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:53.427289 2887 raft_consensus.cc:2468] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:53.427289 2704 raft_consensus.cc:2468] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:53.427804 2763 leader_election.cc:304] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:53.428057 3184 raft_consensus.cc:2804] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:53.428180 3184 raft_consensus.cc:697] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:53.428448 3184 consensus_queue.cc:237] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:53.431424 2586 catalog_manager.cc:5654] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:53.442483 2887 raft_consensus.cc:1275] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:53.442483 2704 raft_consensus.cc:1275] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:53.443122 3198 consensus_queue.cc:1048] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:53.443656 3184 consensus_queue.cc:1048] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:53.444252 2887 raft_consensus.cc:1275] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:53.444665 2807 raft_consensus.cc:1275] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:53.444777 3212 consensus_queue.cc:1048] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:53.445410 3212 consensus_queue.cc:1048] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:52:53.667814 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 148ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:53.928712 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 406ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:54.188892 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 667ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:54.446607 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 927ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:54.462714 2738 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.1:35409, user_credentials={real_user=slave}} blocked reactor thread for 1.00115e+06us
I20251212 08:52:54.463680 2587 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.9"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:54.464568 2587 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.9 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:54.473737 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 0ebda846433246b0af04f3126d32f4ce (DEFAULT_TABLE table=table.9 [id=510e36bc932b421bbfe044399320ad97]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:54.474088 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 0ebda846433246b0af04f3126d32f4ce (DEFAULT_TABLE table=table.9 [id=510e36bc932b421bbfe044399320ad97]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:54.474184 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebda846433246b0af04f3126d32f4ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:54.474291 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 0ebda846433246b0af04f3126d32f4ce (DEFAULT_TABLE table=table.9 [id=510e36bc932b421bbfe044399320ad97]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:54.474393 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebda846433246b0af04f3126d32f4ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:54.474648 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ebda846433246b0af04f3126d32f4ce. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:54.477059 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 6d67e78322704917859cd6c6e62b6964 (DEFAULT_TABLE table=table.9 [id=510e36bc932b421bbfe044399320ad97]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:54.477401 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d67e78322704917859cd6c6e62b6964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:54.481810 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 6d67e78322704917859cd6c6e62b6964 (DEFAULT_TABLE table=table.9 [id=510e36bc932b421bbfe044399320ad97]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:54.482146 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d67e78322704917859cd6c6e62b6964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:54.482371 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 6d67e78322704917859cd6c6e62b6964 (DEFAULT_TABLE table=table.9 [id=510e36bc932b421bbfe044399320ad97]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:54.482833 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6d67e78322704917859cd6c6e62b6964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:54.487646 3236 tablet_bootstrap.cc:492] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:54.489190 3236 tablet_bootstrap.cc:654] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:54.489531 3235 tablet_bootstrap.cc:492] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:54.494320 3235 tablet_bootstrap.cc:654] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:54.497889 3238 tablet_bootstrap.cc:492] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:54.501061 3238 tablet_bootstrap.cc:654] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:54.502278 3235 tablet_bootstrap.cc:492] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:54.502463 3235 ts_tablet_manager.cc:1403] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.000s
I20251212 08:52:54.503149 3235 raft_consensus.cc:359] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:54.503662 3235 raft_consensus.cc:385] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:54.503383 3236 tablet_bootstrap.cc:492] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:54.503791 3235 raft_consensus.cc:740] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:54.503916 3236 ts_tablet_manager.cc:1403] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20251212 08:52:54.504115 3235 consensus_queue.cc:260] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:54.504812 3235 ts_tablet_manager.cc:1434] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:54.504773 3236 raft_consensus.cc:359] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:54.505025 3236 raft_consensus.cc:385] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:54.505147 3236 raft_consensus.cc:740] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:54.505178 3235 tablet_bootstrap.cc:492] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:54.505424 3236 consensus_queue.cc:260] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:54.505936 3236 ts_tablet_manager.cc:1434] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:54.506242 3236 tablet_bootstrap.cc:492] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:54.506924 3235 tablet_bootstrap.cc:654] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:54.507853 3236 tablet_bootstrap.cc:654] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:54.514779 3235 tablet_bootstrap.cc:492] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:54.514982 3238 tablet_bootstrap.cc:492] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:54.514995 3235 ts_tablet_manager.cc:1403] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20251212 08:52:54.515357 3238 ts_tablet_manager.cc:1403] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.000s
I20251212 08:52:54.516005 3238 raft_consensus.cc:359] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:54.516005 3235 raft_consensus.cc:359] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:54.516217 3238 raft_consensus.cc:385] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:54.516307 3235 raft_consensus.cc:385] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:54.516348 3238 raft_consensus.cc:740] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:54.516395 3235 raft_consensus.cc:740] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:54.516563 3238 consensus_queue.cc:260] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:54.516695 3235 consensus_queue.cc:260] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:54.517035 3236 tablet_bootstrap.cc:492] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:54.517074 3238 ts_tablet_manager.cc:1434] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:54.517192 3235 ts_tablet_manager.cc:1434] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251212 08:52:54.517205 3236 ts_tablet_manager.cc:1403] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20251212 08:52:54.517419 3238 tablet_bootstrap.cc:492] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:54.517910 3236 raft_consensus.cc:359] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:54.518139 3236 raft_consensus.cc:385] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:54.518239 3236 raft_consensus.cc:740] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:54.518500 3236 consensus_queue.cc:260] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:54.518846 3238 tablet_bootstrap.cc:654] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:54.519234 3236 ts_tablet_manager.cc:1434] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:54.521071 3238 tablet_bootstrap.cc:492] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:54.521190 3238 ts_tablet_manager.cc:1403] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:54.521900 3238 raft_consensus.cc:359] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:54.522099 3238 raft_consensus.cc:385] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:54.522166 3238 raft_consensus.cc:740] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:54.522352 3238 consensus_queue.cc:260] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:54.522877 3238 ts_tablet_manager.cc:1434] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:54.548633 3227 raft_consensus.cc:493] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:54.548812 3227 raft_consensus.cc:515] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:54.549510 3227 leader_election.cc:290] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:54.549883 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6d67e78322704917859cd6c6e62b6964" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:54.549883 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6d67e78322704917859cd6c6e62b6964" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:54.550091 2885 raft_consensus.cc:2468] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:54.550171 2704 raft_consensus.cc:2468] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:54.550506 2765 leader_election.cc:304] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:54.550798 3227 raft_consensus.cc:2804] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:54.550918 3227 raft_consensus.cc:493] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:54.551008 3227 raft_consensus.cc:3060] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:54.553256 3227 raft_consensus.cc:515] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:54.553958 3227 leader_election.cc:290] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:54.554324 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6d67e78322704917859cd6c6e62b6964" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:54.554322 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "6d67e78322704917859cd6c6e62b6964" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:54.554541 2885 raft_consensus.cc:3060] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:54.554541 2704 raft_consensus.cc:3060] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:54.557448 2885 raft_consensus.cc:2468] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:54.557448 2704 raft_consensus.cc:2468] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:54.557878 2765 leader_election.cc:304] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:54.558180 3227 raft_consensus.cc:2804] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:54.558333 3227 raft_consensus.cc:697] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:54.558568 3227 consensus_queue.cc:237] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:54.561453 2587 catalog_manager.cc:5654] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:54.562310 3219 raft_consensus.cc:493] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:54.562467 3219 raft_consensus.cc:515] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:54.563066 3219 leader_election.cc:290] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:54.563393 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0ebda846433246b0af04f3126d32f4ce" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:54.563563 2807 raft_consensus.cc:2468] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:54.563522 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0ebda846433246b0af04f3126d32f4ce" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:54.563683 2704 raft_consensus.cc:2468] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:54.564016 2843 leader_election.cc:304] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:54.564267 3219 raft_consensus.cc:2804] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:54.564381 3219 raft_consensus.cc:493] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:54.564473 3219 raft_consensus.cc:3060] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:54.566402 3219 raft_consensus.cc:515] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:54.566859 3219 leader_election.cc:290] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:54.567258 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0ebda846433246b0af04f3126d32f4ce" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:54.567276 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0ebda846433246b0af04f3126d32f4ce" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:54.567406 2704 raft_consensus.cc:3060] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:54.567446 2807 raft_consensus.cc:3060] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:54.569758 2704 raft_consensus.cc:2468] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:54.569773 2807 raft_consensus.cc:2468] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:54.570101 2843 leader_election.cc:304] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:54.570348 3219 raft_consensus.cc:2804] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:54.570731 3219 raft_consensus.cc:697] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:52:54.571031 3219 consensus_queue.cc:237] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:54.574069 2587 catalog_manager.cc:5654] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:54.597440 2704 raft_consensus.cc:1275] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:54.597452 2807 raft_consensus.cc:1275] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:54.598031 3219 consensus_queue.cc:1048] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:54.598742 2885 raft_consensus.cc:1275] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:54.598742 2702 raft_consensus.cc:1275] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:54.599367 3227 consensus_queue.cc:1048] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:54.599589 3244 consensus_queue.cc:1048] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:54.599722 3219 consensus_queue.cc:1048] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:54.628372 3248 mvcc.cc:204] Tried to move back new op lower bound from 7231609137545486336 to 7231609137394409472. Current Snapshot: MvccSnapshot[applied={T|T < 7231609137545486336 or (T in {7231609137545486336})}]
W20251212 08:52:54.904089 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 223ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:55.166901 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 480ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:55.426558 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 743ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:55.625347 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.3:33005, user_credentials={real_user=slave}} blocked reactor thread for 1.00094e+06us
I20251212 08:52:55.626976 2586 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.10"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:55.627875 2586 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.10 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:55.637100 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 881346b54fb94223a5246c0c38a20d91 (DEFAULT_TABLE table=table.10 [id=1bb690012fc644c7bdcabb32843dc675]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:55.637590 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 881346b54fb94223a5246c0c38a20d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:55.637728 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 881346b54fb94223a5246c0c38a20d91 (DEFAULT_TABLE table=table.10 [id=1bb690012fc644c7bdcabb32843dc675]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:55.638044 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 881346b54fb94223a5246c0c38a20d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:55.641460 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 881346b54fb94223a5246c0c38a20d91 (DEFAULT_TABLE table=table.10 [id=1bb690012fc644c7bdcabb32843dc675]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:55.644650 3272 tablet_bootstrap.cc:492] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:55.646071 3272 tablet_bootstrap.cc:654] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:55.646945 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 35dec88dbd394d4a80ee2e863aec6bbf (DEFAULT_TABLE table=table.10 [id=1bb690012fc644c7bdcabb32843dc675]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:55.647262 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35dec88dbd394d4a80ee2e863aec6bbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:55.652726 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 881346b54fb94223a5246c0c38a20d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:55.653962 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 35dec88dbd394d4a80ee2e863aec6bbf (DEFAULT_TABLE table=table.10 [id=1bb690012fc644c7bdcabb32843dc675]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:55.654320 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35dec88dbd394d4a80ee2e863aec6bbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:55.664018 3272 tablet_bootstrap.cc:492] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:55.664227 3272 ts_tablet_manager.cc:1403] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.005s
I20251212 08:52:55.665320 3272 raft_consensus.cc:359] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:55.665597 3272 raft_consensus.cc:385] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:55.666141 3274 tablet_bootstrap.cc:492] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:55.665697 3272 raft_consensus.cc:740] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:55.667550 3272 consensus_queue.cc:260] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:55.668267 3274 tablet_bootstrap.cc:654] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:55.669296 3275 tablet_bootstrap.cc:492] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:55.670238 3272 ts_tablet_manager.cc:1434] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s
I20251212 08:52:55.670853 3275 tablet_bootstrap.cc:654] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:55.672498 3274 tablet_bootstrap.cc:492] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:55.672686 3274 ts_tablet_manager.cc:1403] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.002s
I20251212 08:52:55.672843 3275 tablet_bootstrap.cc:492] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:55.673039 3275 ts_tablet_manager.cc:1403] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:52:55.673054 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 35dec88dbd394d4a80ee2e863aec6bbf (DEFAULT_TABLE table=table.10 [id=1bb690012fc644c7bdcabb32843dc675]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:55.673997 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35dec88dbd394d4a80ee2e863aec6bbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:55.674115 3274 raft_consensus.cc:359] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:55.674292 3274 raft_consensus.cc:385] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:55.674441 3274 raft_consensus.cc:740] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:55.674705 3274 consensus_queue.cc:260] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:55.675480 3274 ts_tablet_manager.cc:1434] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:52:55.675750 3274 tablet_bootstrap.cc:492] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:55.676947 3275 raft_consensus.cc:359] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:55.677136 3274 tablet_bootstrap.cc:654] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:55.677124 3275 raft_consensus.cc:385] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:55.677881 3275 raft_consensus.cc:740] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:55.678098 3275 consensus_queue.cc:260] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:55.678571 3275 ts_tablet_manager.cc:1434] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251212 08:52:55.678838 3275 tablet_bootstrap.cc:492] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:55.679926 3272 tablet_bootstrap.cc:492] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:55.680159 3275 tablet_bootstrap.cc:654] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:55.680480 3274 tablet_bootstrap.cc:492] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:55.680617 3274 ts_tablet_manager.cc:1403] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20251212 08:52:55.681581 3272 tablet_bootstrap.cc:654] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:55.682358 3274 raft_consensus.cc:359] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:55.682662 3274 raft_consensus.cc:385] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:55.682780 3274 raft_consensus.cc:740] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:55.683117 3274 consensus_queue.cc:260] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:55.685166 3274 ts_tablet_manager.cc:1434] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:52:55.689414 3275 tablet_bootstrap.cc:492] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:55.689558 3275 ts_tablet_manager.cc:1403] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.006s
I20251212 08:52:55.690132 3275 raft_consensus.cc:359] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:55.690269 3275 raft_consensus.cc:385] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:55.690325 3275 raft_consensus.cc:740] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:55.690529 3275 consensus_queue.cc:260] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:55.691068 3275 ts_tablet_manager.cc:1434] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251212 08:52:55.697436 3272 tablet_bootstrap.cc:492] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:55.697597 3272 ts_tablet_manager.cc:1403] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.018s user 0.007s sys 0.008s
I20251212 08:52:55.698176 3272 raft_consensus.cc:359] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:55.698325 3272 raft_consensus.cc:385] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:55.698387 3272 raft_consensus.cc:740] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:55.698570 3272 consensus_queue.cc:260] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:55.699155 3272 ts_tablet_manager.cc:1434] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251212 08:52:55.722466 3264 raft_consensus.cc:493] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:55.722658 3264 raft_consensus.cc:515] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:55.723321 3264 leader_election.cc:290] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:52:55.723595 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35dec88dbd394d4a80ee2e863aec6bbf" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:55.723747 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35dec88dbd394d4a80ee2e863aec6bbf" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:55.723793 2702 raft_consensus.cc:2468] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:55.723877 2807 raft_consensus.cc:2468] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:52:55.724181 2843 leader_election.cc:304] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:55.724450 3264 raft_consensus.cc:2804] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:55.724524 3264 raft_consensus.cc:493] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:55.724568 3264 raft_consensus.cc:3060] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:55.726516 3264 raft_consensus.cc:515] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:55.727241 3264 leader_election.cc:290] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:52:55.727355 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35dec88dbd394d4a80ee2e863aec6bbf" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:55.727629 2702 raft_consensus.cc:3060] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:55.727371 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "35dec88dbd394d4a80ee2e863aec6bbf" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:55.727816 2807 raft_consensus.cc:3060] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:55.729821 2702 raft_consensus.cc:2468] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:55.729900 2807 raft_consensus.cc:2468] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:52:55.730137 2843 leader_election.cc:304] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:55.730376 3264 raft_consensus.cc:2804] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:55.730482 3264 raft_consensus.cc:697] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:52:55.730710 3264 consensus_queue.cc:237] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:55.733374 2585 catalog_manager.cc:5654] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:55.775384 3212 raft_consensus.cc:493] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:55.775552 3212 raft_consensus.cc:515] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:55.776134 3212 leader_election.cc:290] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:55.776594 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "881346b54fb94223a5246c0c38a20d91" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:55.776685 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "881346b54fb94223a5246c0c38a20d91" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:55.776783 2807 raft_consensus.cc:2468] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:55.776942 2885 raft_consensus.cc:2468] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:55.777191 2667 leader_election.cc:304] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:55.777427 3212 raft_consensus.cc:2804] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:55.777511 3212 raft_consensus.cc:493] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:55.777599 3212 raft_consensus.cc:3060] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:55.779927 3212 raft_consensus.cc:515] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:55.780467 3212 leader_election.cc:290] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:55.780835 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "881346b54fb94223a5246c0c38a20d91" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:55.780869 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "881346b54fb94223a5246c0c38a20d91" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:55.780978 2807 raft_consensus.cc:3060] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:55.781005 2885 raft_consensus.cc:3060] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:55.782831 2807 raft_consensus.cc:2468] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:55.782838 2885 raft_consensus.cc:2468] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:55.783178 2667 leader_election.cc:304] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:55.783377 3212 raft_consensus.cc:2804] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:55.783488 3212 raft_consensus.cc:697] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:52:55.783716 3212 consensus_queue.cc:237] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:55.786703 2585 catalog_manager.cc:5654] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } }
I20251212 08:52:55.804543 2885 raft_consensus.cc:1275] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:55.804633 2807 raft_consensus.cc:1275] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:55.805114 3267 consensus_queue.cc:1048] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:55.805301 3212 consensus_queue.cc:1048] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:55.819610 2702 raft_consensus.cc:1275] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:55.820343 3264 consensus_queue.cc:1048] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:55.827180 2807 raft_consensus.cc:1275] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:55.827663 3264 consensus_queue.cc:1048] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:52:56.087184 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 206ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:56.345820 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 465ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:56.603313 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 724ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:56.823261 2738 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.1:35409, user_credentials={real_user=slave}} blocked reactor thread for 1.00115e+06us
I20251212 08:52:56.824229 2589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.11"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:56.826475 2589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.11 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:56.841517 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 402f26e2e46f416cb4ee88c2f09a5dfd (DEFAULT_TABLE table=table.11 [id=8cf5be8d27464865a5de9a661fbe569f]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:56.842007 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402f26e2e46f416cb4ee88c2f09a5dfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:56.842860 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 402f26e2e46f416cb4ee88c2f09a5dfd (DEFAULT_TABLE table=table.11 [id=8cf5be8d27464865a5de9a661fbe569f]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:56.843284 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402f26e2e46f416cb4ee88c2f09a5dfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:56.843506 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 402f26e2e46f416cb4ee88c2f09a5dfd (DEFAULT_TABLE table=table.11 [id=8cf5be8d27464865a5de9a661fbe569f]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:56.843845 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 402f26e2e46f416cb4ee88c2f09a5dfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:56.848659 2694 tablet_service.cc:1505] Processing CreateTablet for tablet c40e9cbf2b184cb7be83ed003ecadfde (DEFAULT_TABLE table=table.11 [id=8cf5be8d27464865a5de9a661fbe569f]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:56.849016 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c40e9cbf2b184cb7be83ed003ecadfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:56.849097 2873 tablet_service.cc:1505] Processing CreateTablet for tablet c40e9cbf2b184cb7be83ed003ecadfde (DEFAULT_TABLE table=table.11 [id=8cf5be8d27464865a5de9a661fbe569f]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:56.849489 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c40e9cbf2b184cb7be83ed003ecadfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:56.853397 2793 tablet_service.cc:1505] Processing CreateTablet for tablet c40e9cbf2b184cb7be83ed003ecadfde (DEFAULT_TABLE table=table.11 [id=8cf5be8d27464865a5de9a661fbe569f]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:56.853920 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c40e9cbf2b184cb7be83ed003ecadfde. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:56.858497 3301 tablet_bootstrap.cc:492] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:56.863283 3301 tablet_bootstrap.cc:654] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:56.872781 3300 tablet_bootstrap.cc:492] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:56.875078 3300 tablet_bootstrap.cc:654] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:56.875306 3303 tablet_bootstrap.cc:492] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:56.879912 3303 tablet_bootstrap.cc:654] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:56.891287 3300 tablet_bootstrap.cc:492] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:56.891448 3300 ts_tablet_manager.cc:1403] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.004s
I20251212 08:52:56.891783 3301 tablet_bootstrap.cc:492] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:56.891942 3301 ts_tablet_manager.cc:1403] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.034s user 0.004s sys 0.009s
I20251212 08:52:56.892318 3300 raft_consensus.cc:359] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:56.892490 3300 raft_consensus.cc:385] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:56.892565 3300 raft_consensus.cc:740] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:56.892786 3300 consensus_queue.cc:260] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:56.892928 3301 raft_consensus.cc:359] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:56.893186 3301 raft_consensus.cc:385] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:56.893256 3301 raft_consensus.cc:740] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:56.893451 3301 consensus_queue.cc:260] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:56.893333 3300 ts_tablet_manager.cc:1434] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:56.893971 3300 tablet_bootstrap.cc:492] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:56.894053 3301 ts_tablet_manager.cc:1434] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251212 08:52:56.894315 3301 tablet_bootstrap.cc:492] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:56.897218 3303 tablet_bootstrap.cc:492] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:56.897372 3303 ts_tablet_manager.cc:1403] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.000s
I20251212 08:52:56.898236 3300 tablet_bootstrap.cc:654] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:56.898200 3303 raft_consensus.cc:359] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:56.898432 3303 raft_consensus.cc:385] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:56.898536 3303 raft_consensus.cc:740] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:56.898772 3303 consensus_queue.cc:260] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:56.900064 3303 ts_tablet_manager.cc:1434] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:52:56.900358 3300 tablet_bootstrap.cc:492] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:56.900585 3300 ts_tablet_manager.cc:1403] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251212 08:52:56.900652 3301 tablet_bootstrap.cc:654] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:56.900998 3303 tablet_bootstrap.cc:492] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:56.902666 3303 tablet_bootstrap.cc:654] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:56.902717 3301 tablet_bootstrap.cc:492] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:56.902890 3301 ts_tablet_manager.cc:1403] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.000s
I20251212 08:52:56.903680 3301 raft_consensus.cc:359] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:56.903826 3301 raft_consensus.cc:385] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:56.903883 3301 raft_consensus.cc:740] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:56.904074 3301 consensus_queue.cc:260] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:56.904664 3301 ts_tablet_manager.cc:1434] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:56.905750 3300 raft_consensus.cc:359] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:56.906023 3300 raft_consensus.cc:385] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:56.906158 3300 raft_consensus.cc:740] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:56.906486 3303 tablet_bootstrap.cc:492] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:56.906666 3303 ts_tablet_manager.cc:1403] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20251212 08:52:56.906477 3300 consensus_queue.cc:260] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:56.907339 3300 ts_tablet_manager.cc:1434] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:56.907490 3303 raft_consensus.cc:359] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:56.907680 3303 raft_consensus.cc:385] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:56.907768 3303 raft_consensus.cc:740] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:56.907984 3303 consensus_queue.cc:260] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:56.908674 3303 ts_tablet_manager.cc:1434] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:57.073770 3212 raft_consensus.cc:493] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:57.073951 3212 raft_consensus.cc:515] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:57.074762 3212 leader_election.cc:290] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:57.075277 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c40e9cbf2b184cb7be83ed003ecadfde" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:57.075423 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c40e9cbf2b184cb7be83ed003ecadfde" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:57.075513 2885 raft_consensus.cc:2468] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:57.075641 2807 raft_consensus.cc:2468] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:57.076040 2667 leader_election.cc:304] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:57.076345 3212 raft_consensus.cc:2804] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:57.076453 3212 raft_consensus.cc:493] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:57.076586 3212 raft_consensus.cc:3060] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:57.080926 3212 raft_consensus.cc:515] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:57.081593 3212 leader_election.cc:290] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:57.081979 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c40e9cbf2b184cb7be83ed003ecadfde" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:57.081979 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c40e9cbf2b184cb7be83ed003ecadfde" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:57.082186 2807 raft_consensus.cc:3060] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:57.082216 2885 raft_consensus.cc:3060] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:57.085372 2807 raft_consensus.cc:2468] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:57.085372 2885 raft_consensus.cc:2468] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:57.085844 2667 leader_election.cc:304] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:57.086159 3212 raft_consensus.cc:2804] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:57.086284 3212 raft_consensus.cc:697] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:52:57.086599 3212 consensus_queue.cc:237] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:57.089610 2586 catalog_manager.cc:5654] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:57.133832 3266 raft_consensus.cc:493] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:57.134028 3266 raft_consensus.cc:515] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:57.134658 3266 leader_election.cc:290] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:57.135069 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "402f26e2e46f416cb4ee88c2f09a5dfd" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:57.135071 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "402f26e2e46f416cb4ee88c2f09a5dfd" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:57.135412 2702 raft_consensus.cc:2468] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:57.135414 2885 raft_consensus.cc:2468] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:57.135893 2763 leader_election.cc:304] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:57.136162 3266 raft_consensus.cc:2804] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:57.136271 3266 raft_consensus.cc:493] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:57.136402 3266 raft_consensus.cc:3060] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:57.138463 3266 raft_consensus.cc:515] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:57.138978 3266 leader_election.cc:290] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:57.139348 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "402f26e2e46f416cb4ee88c2f09a5dfd" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:57.139397 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "402f26e2e46f416cb4ee88c2f09a5dfd" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:57.139575 2885 raft_consensus.cc:3060] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:57.139597 2702 raft_consensus.cc:3060] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:57.142795 2702 raft_consensus.cc:2468] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:57.142795 2885 raft_consensus.cc:2468] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:57.143246 2765 leader_election.cc:304] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:52:57.143559 3266 raft_consensus.cc:2804] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:57.143752 3266 raft_consensus.cc:697] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:57.144088 3266 consensus_queue.cc:237] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:57.146847 2586 catalog_manager.cc:5654] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:57.226269 2885 raft_consensus.cc:1275] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:57.226300 2807 raft_consensus.cc:1275] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:57.226869 3309 consensus_queue.cc:1048] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:57.227277 3309 consensus_queue.cc:1048] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:57.228094 2702 raft_consensus.cc:1275] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:57.228716 3311 consensus_queue.cc:1048] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:57.237087 2885 raft_consensus.cc:1275] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:57.237675 3311 consensus_queue.cc:1048] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:52:57.553682 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 249ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:57.813323 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 509ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:58.070006 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 768ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:58.246124 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00098e+06us
I20251212 08:52:58.247318 2589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.12"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:58.248189 2589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.12 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:58.257434 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 9f0b0e1a566f4bea899323654c7248a9 (DEFAULT_TABLE table=table.12 [id=0aa9dcfd42e9422880f97180138de37e]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:58.257885 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f0b0e1a566f4bea899323654c7248a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:58.257805 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 9f0b0e1a566f4bea899323654c7248a9 (DEFAULT_TABLE table=table.12 [id=0aa9dcfd42e9422880f97180138de37e]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:58.258168 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f0b0e1a566f4bea899323654c7248a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:58.259397 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 9f0b0e1a566f4bea899323654c7248a9 (DEFAULT_TABLE table=table.12 [id=0aa9dcfd42e9422880f97180138de37e]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:58.259477 2695 tablet_service.cc:1505] Processing CreateTablet for tablet d84415e201b34feb9991ac04c8355802 (DEFAULT_TABLE table=table.12 [id=0aa9dcfd42e9422880f97180138de37e]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:58.259766 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f0b0e1a566f4bea899323654c7248a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:58.259991 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d84415e201b34feb9991ac04c8355802. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:58.264549 2795 tablet_service.cc:1505] Processing CreateTablet for tablet d84415e201b34feb9991ac04c8355802 (DEFAULT_TABLE table=table.12 [id=0aa9dcfd42e9422880f97180138de37e]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:58.264915 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d84415e201b34feb9991ac04c8355802. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:58.265058 2876 tablet_service.cc:1505] Processing CreateTablet for tablet d84415e201b34feb9991ac04c8355802 (DEFAULT_TABLE table=table.12 [id=0aa9dcfd42e9422880f97180138de37e]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:58.265385 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d84415e201b34feb9991ac04c8355802. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:58.277491 3331 tablet_bootstrap.cc:492] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:58.278788 3332 tablet_bootstrap.cc:492] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:58.279345 3331 tablet_bootstrap.cc:654] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:58.280563 3332 tablet_bootstrap.cc:654] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:58.287458 3331 tablet_bootstrap.cc:492] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:58.287701 3331 ts_tablet_manager.cc:1403] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.003s
I20251212 08:52:58.288589 3331 raft_consensus.cc:359] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:58.288894 3331 raft_consensus.cc:385] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:58.289012 3331 raft_consensus.cc:740] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:58.289345 3331 consensus_queue.cc:260] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:58.289966 3331 ts_tablet_manager.cc:1434] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:58.291131 3331 tablet_bootstrap.cc:492] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:58.291365 3332 tablet_bootstrap.cc:492] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:58.291536 3332 ts_tablet_manager.cc:1403] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20251212 08:52:58.291872 3333 tablet_bootstrap.cc:492] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:58.292414 3332 raft_consensus.cc:359] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:58.292647 3332 raft_consensus.cc:385] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:58.292757 3332 raft_consensus.cc:740] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:58.292938 3331 tablet_bootstrap.cc:654] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:58.293066 3332 consensus_queue.cc:260] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:58.293995 3333 tablet_bootstrap.cc:654] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:58.297175 3332 ts_tablet_manager.cc:1434] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20251212 08:52:58.297501 3332 tablet_bootstrap.cc:492] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:58.297889 3333 tablet_bootstrap.cc:492] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:58.297880 3331 tablet_bootstrap.cc:492] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:58.298151 3331 ts_tablet_manager.cc:1403] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20251212 08:52:58.298161 3333 ts_tablet_manager.cc:1403] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20251212 08:52:58.299026 3331 raft_consensus.cc:359] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:58.299280 3331 raft_consensus.cc:385] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:58.299369 3331 raft_consensus.cc:740] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:58.299597 3331 consensus_queue.cc:260] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:58.300367 3331 ts_tablet_manager.cc:1434] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:58.299952 3333 raft_consensus.cc:359] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:58.301141 3333 raft_consensus.cc:385] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:58.301218 3333 raft_consensus.cc:740] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:58.301256 3332 tablet_bootstrap.cc:654] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:58.301411 3333 consensus_queue.cc:260] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:58.302328 3333 ts_tablet_manager.cc:1434] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:52:58.302798 3333 tablet_bootstrap.cc:492] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:58.304070 3333 tablet_bootstrap.cc:654] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:58.306406 3332 tablet_bootstrap.cc:492] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:58.306787 3332 ts_tablet_manager.cc:1403] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20251212 08:52:58.307322 3333 tablet_bootstrap.cc:492] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:58.307513 3333 ts_tablet_manager.cc:1403] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251212 08:52:58.307631 3332 raft_consensus.cc:359] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:58.307842 3332 raft_consensus.cc:385] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:58.307940 3332 raft_consensus.cc:740] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:58.308207 3332 consensus_queue.cc:260] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:58.308859 3332 ts_tablet_manager.cc:1434] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:58.308581 3333 raft_consensus.cc:359] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:58.309057 3333 raft_consensus.cc:385] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:58.309126 3333 raft_consensus.cc:740] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:58.309324 3333 consensus_queue.cc:260] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:58.309774 3333 ts_tablet_manager.cc:1434] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:58.495822 3311 raft_consensus.cc:493] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:58.495980 3311 raft_consensus.cc:515] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:58.496712 3311 leader_election.cc:290] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:58.497108 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f0b0e1a566f4bea899323654c7248a9" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:58.497133 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f0b0e1a566f4bea899323654c7248a9" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:52:58.497385 2702 raft_consensus.cc:2468] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:58.497385 2885 raft_consensus.cc:2468] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:52:58.497871 2763 leader_election.cc:304] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:58.498140 3311 raft_consensus.cc:2804] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:58.498229 3311 raft_consensus.cc:493] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:58.498317 3311 raft_consensus.cc:3060] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:58.500456 3311 raft_consensus.cc:515] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:58.500926 3311 leader_election.cc:290] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:52:58.501307 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f0b0e1a566f4bea899323654c7248a9" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:58.501307 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f0b0e1a566f4bea899323654c7248a9" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:52:58.501487 2885 raft_consensus.cc:3060] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:58.501487 2702 raft_consensus.cc:3060] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:58.504254 2702 raft_consensus.cc:2468] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:58.504254 2885 raft_consensus.cc:2468] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:52:58.504796 2763 leader_election.cc:304] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:58.505048 3311 raft_consensus.cc:2804] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:58.505189 3311 raft_consensus.cc:697] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:52:58.505424 3311 consensus_queue.cc:237] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:58.508180 2586 catalog_manager.cc:5654] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:58.566282 3323 raft_consensus.cc:493] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:52:58.566444 3323 raft_consensus.cc:515] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:58.567268 3323 leader_election.cc:290] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:58.567713 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d84415e201b34feb9991ac04c8355802" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:52:58.567766 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d84415e201b34feb9991ac04c8355802" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:52:58.567879 2807 raft_consensus.cc:2468] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:58.567924 2885 raft_consensus.cc:2468] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:52:58.568333 2667 leader_election.cc:304] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:58.568570 3323 raft_consensus.cc:2804] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:52:58.568634 3323 raft_consensus.cc:493] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:52:58.568694 3323 raft_consensus.cc:3060] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:58.570526 3323 raft_consensus.cc:515] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:58.570958 3323 leader_election.cc:290] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:52:58.571450 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d84415e201b34feb9991ac04c8355802" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:52:58.571532 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d84415e201b34feb9991ac04c8355802" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:52:58.571599 2807 raft_consensus.cc:3060] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:58.571662 2885 raft_consensus.cc:3060] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:52:58.574359 2807 raft_consensus.cc:2468] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:58.574438 2885 raft_consensus.cc:2468] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:52:58.574748 2667 leader_election.cc:304] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:52:58.574995 3323 raft_consensus.cc:2804] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:52:58.575163 3323 raft_consensus.cc:697] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:52:58.575423 3323 consensus_queue.cc:237] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:58.579749 2586 catalog_manager.cc:5654] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:52:58.640837 2702 raft_consensus.cc:1275] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:58.640952 2885 raft_consensus.cc:1275] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:58.641595 3311 consensus_queue.cc:1048] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:58.641856 3324 consensus_queue.cc:1048] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:58.642059 2807 raft_consensus.cc:1275] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:58.641866 2885 raft_consensus.cc:1275] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:52:58.642585 3326 consensus_queue.cc:1048] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:52:58.645265 3323 consensus_queue.cc:1048] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:52:58.937384 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 189ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:59.201896 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 446ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:59.461598 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 711ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:52:59.692529 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00116e+06us
I20251212 08:52:59.693495 2589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.13"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:52:59.694564 2589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.13 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:52:59.707872 2793 tablet_service.cc:1505] Processing CreateTablet for tablet a295a2fa31fe405da6380ccfd082338b (DEFAULT_TABLE table=table.13 [id=15821162733f42f7a27b5b1eb26b1439]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:59.708343 2695 tablet_service.cc:1505] Processing CreateTablet for tablet a295a2fa31fe405da6380ccfd082338b (DEFAULT_TABLE table=table.13 [id=15821162733f42f7a27b5b1eb26b1439]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:59.708472 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a295a2fa31fe405da6380ccfd082338b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:59.708777 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a295a2fa31fe405da6380ccfd082338b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:59.712078 2876 tablet_service.cc:1505] Processing CreateTablet for tablet a295a2fa31fe405da6380ccfd082338b (DEFAULT_TABLE table=table.13 [id=15821162733f42f7a27b5b1eb26b1439]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:52:59.712563 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a295a2fa31fe405da6380ccfd082338b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:59.713589 2694 tablet_service.cc:1505] Processing CreateTablet for tablet d0771ee252414935aabcc7a9aa6e922f (DEFAULT_TABLE table=table.13 [id=15821162733f42f7a27b5b1eb26b1439]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:59.714010 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0771ee252414935aabcc7a9aa6e922f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:59.715757 2795 tablet_service.cc:1505] Processing CreateTablet for tablet d0771ee252414935aabcc7a9aa6e922f (DEFAULT_TABLE table=table.13 [id=15821162733f42f7a27b5b1eb26b1439]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:59.715847 2873 tablet_service.cc:1505] Processing CreateTablet for tablet d0771ee252414935aabcc7a9aa6e922f (DEFAULT_TABLE table=table.13 [id=15821162733f42f7a27b5b1eb26b1439]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:52:59.716145 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0771ee252414935aabcc7a9aa6e922f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:59.717804 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0771ee252414935aabcc7a9aa6e922f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:52:59.734891 3366 tablet_bootstrap.cc:492] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:59.737919 3366 tablet_bootstrap.cc:654] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:59.743258 3367 tablet_bootstrap.cc:492] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:59.744437 3365 tablet_bootstrap.cc:492] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:59.744648 3367 tablet_bootstrap.cc:654] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:59.746517 3367 tablet_bootstrap.cc:492] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:59.746663 3367 ts_tablet_manager.cc:1403] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:52:59.747787 3365 tablet_bootstrap.cc:654] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:59.749831 3367 raft_consensus.cc:359] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:59.750063 3367 raft_consensus.cc:385] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:59.750149 3367 raft_consensus.cc:740] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:59.750415 3367 consensus_queue.cc:260] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:59.751122 3367 ts_tablet_manager.cc:1434] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:52:59.751571 3367 tablet_bootstrap.cc:492] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:52:59.754925 3367 tablet_bootstrap.cc:654] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:59.759361 3366 tablet_bootstrap.cc:492] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:59.759548 3366 ts_tablet_manager.cc:1403] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.000s
I20251212 08:52:59.759654 3367 tablet_bootstrap.cc:492] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:52:59.760493 3366 raft_consensus.cc:359] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:59.760735 3366 raft_consensus.cc:385] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:59.760819 3366 raft_consensus.cc:740] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:59.761749 3367 ts_tablet_manager.cc:1403] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.004s
I20251212 08:52:59.763099 3367 raft_consensus.cc:359] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:59.763451 3367 raft_consensus.cc:385] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:59.763572 3367 raft_consensus.cc:740] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:52:59.764134 3367 consensus_queue.cc:260] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:59.765102 3367 ts_tablet_manager.cc:1434] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:52:59.768705 3366 consensus_queue.cc:260] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:59.771328 3366 ts_tablet_manager.cc:1434] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.012s user 0.000s sys 0.003s
I20251212 08:52:59.772941 3366 tablet_bootstrap.cc:492] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:52:59.775206 3366 tablet_bootstrap.cc:654] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:59.777992 3366 tablet_bootstrap.cc:492] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:52:59.778162 3366 ts_tablet_manager.cc:1403] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20251212 08:52:59.779029 3366 raft_consensus.cc:359] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:59.779258 3366 raft_consensus.cc:385] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:59.779356 3365 tablet_bootstrap.cc:492] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:59.779350 3366 raft_consensus.cc:740] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:52:59.779510 3365 ts_tablet_manager.cc:1403] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.035s user 0.000s sys 0.023s
I20251212 08:52:59.779595 3366 consensus_queue.cc:260] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:59.780237 3366 ts_tablet_manager.cc:1434] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251212 08:52:59.780275 3365 raft_consensus.cc:359] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:59.780455 3365 raft_consensus.cc:385] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:59.780515 3365 raft_consensus.cc:740] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:59.780725 3365 consensus_queue.cc:260] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:52:59.781306 3365 ts_tablet_manager.cc:1434] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:52:59.781687 3365 tablet_bootstrap.cc:492] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:52:59.783208 3365 tablet_bootstrap.cc:654] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:52:59.786520 3365 tablet_bootstrap.cc:492] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:52:59.786739 3365 ts_tablet_manager.cc:1403] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251212 08:52:59.787422 3365 raft_consensus.cc:359] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:59.787588 3365 raft_consensus.cc:385] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:52:59.787672 3365 raft_consensus.cc:740] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:52:59.787899 3365 consensus_queue.cc:260] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:52:59.788481 3365 ts_tablet_manager.cc:1434] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:52:59.999704 3358 raft_consensus.cc:493] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:00.000029 3358 raft_consensus.cc:515] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:00.000677 3358 leader_election.cc:290] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:00.001179 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0771ee252414935aabcc7a9aa6e922f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:00.001243 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0771ee252414935aabcc7a9aa6e922f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:00.001427 2807 raft_consensus.cc:2468] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:00.001433 2704 raft_consensus.cc:2468] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:00.001976 2843 leader_election.cc:304] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:00.002250 3358 raft_consensus.cc:2804] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:00.002336 3358 raft_consensus.cc:493] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:00.002439 3358 raft_consensus.cc:3060] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:00.021255 3358 raft_consensus.cc:515] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:00.022238 3358 leader_election.cc:290] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:00.022666 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0771ee252414935aabcc7a9aa6e922f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:00.022704 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d0771ee252414935aabcc7a9aa6e922f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:00.022951 2704 raft_consensus.cc:3060] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:00.023039 2807 raft_consensus.cc:3060] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:00.025768 2704 raft_consensus.cc:2468] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:00.026383 2843 leader_election.cc:304] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:00.026598 2807 raft_consensus.cc:2468] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:00.027179 3358 raft_consensus.cc:2804] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:00.027329 3358 raft_consensus.cc:697] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:00.027595 3358 consensus_queue.cc:237] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:00.031330 2586 catalog_manager.cc:5654] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:00.082140 3359 raft_consensus.cc:493] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:00.082345 3359 raft_consensus.cc:515] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:00.083031 3359 leader_election.cc:290] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:00.083849 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a2fa31fe405da6380ccfd082338b" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:00.084025 2886 raft_consensus.cc:2468] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:00.085043 2765 leader_election.cc:304] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:00.085453 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a2fa31fe405da6380ccfd082338b" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:00.085631 2704 raft_consensus.cc:2468] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:00.086508 3359 raft_consensus.cc:2804] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:00.086601 3359 raft_consensus.cc:493] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:00.086683 3359 raft_consensus.cc:3060] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:00.120316 3359 raft_consensus.cc:515] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:00.121290 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a2fa31fe405da6380ccfd082338b" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:00.121440 3359 leader_election.cc:290] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:00.121474 2885 raft_consensus.cc:3060] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:00.124253 2885 raft_consensus.cc:2468] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:00.124696 2765 leader_election.cc:304] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:00.124958 3359 raft_consensus.cc:2804] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:00.125095 3359 raft_consensus.cc:697] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:00.125351 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a295a2fa31fe405da6380ccfd082338b" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:00.125535 2704 raft_consensus.cc:3060] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:00.125351 3359 consensus_queue.cc:237] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:00.132370 2589 catalog_manager.cc:5654] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:00.181442 2807 raft_consensus.cc:1275] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:00.182031 2705 raft_consensus.cc:1275] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:00.182031 2885 raft_consensus.cc:1275] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:00.182068 3358 consensus_queue.cc:1048] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:00.182545 3374 consensus_queue.cc:1048] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:00.182813 3359 consensus_queue.cc:1048] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:00.192224 3378 mvcc.cc:204] Tried to move back new op lower bound from 7231609160418668544 to 7231609159796117504. Current Snapshot: MvccSnapshot[applied={T|T < 7231609160418668544}]
I20251212 08:53:00.223621 2704 raft_consensus.cc:2468] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:00.224004 2702 raft_consensus.cc:1275] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:00.224974 3357 consensus_queue.cc:1048] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:00.438576 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 183ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:00.705757 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 444ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:00.967061 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 711ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:01.195561 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.3:33005, user_credentials={real_user=slave}} blocked reactor thread for 1.00115e+06us
I20251212 08:53:01.197676 2586 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.14"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:01.198519 2586 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.14 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:01.211011 2795 tablet_service.cc:1505] Processing CreateTablet for tablet bc238dfc34fc4acb9d646c80e38c01b9 (DEFAULT_TABLE table=table.14 [id=0a995bdfd71c4ddb9a3222d9c77c7bbb]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:01.211531 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc238dfc34fc4acb9d646c80e38c01b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:01.213271 2694 tablet_service.cc:1505] Processing CreateTablet for tablet dfc6271b43244577a6b05d092920f236 (DEFAULT_TABLE table=table.14 [id=0a995bdfd71c4ddb9a3222d9c77c7bbb]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:01.213356 2876 tablet_service.cc:1505] Processing CreateTablet for tablet dfc6271b43244577a6b05d092920f236 (DEFAULT_TABLE table=table.14 [id=0a995bdfd71c4ddb9a3222d9c77c7bbb]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:01.213385 2873 tablet_service.cc:1505] Processing CreateTablet for tablet bc238dfc34fc4acb9d646c80e38c01b9 (DEFAULT_TABLE table=table.14 [id=0a995bdfd71c4ddb9a3222d9c77c7bbb]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:01.213692 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc6271b43244577a6b05d092920f236. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:01.213846 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc6271b43244577a6b05d092920f236. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:01.214207 2695 tablet_service.cc:1505] Processing CreateTablet for tablet bc238dfc34fc4acb9d646c80e38c01b9 (DEFAULT_TABLE table=table.14 [id=0a995bdfd71c4ddb9a3222d9c77c7bbb]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:01.214553 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc238dfc34fc4acb9d646c80e38c01b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:01.215566 2793 tablet_service.cc:1505] Processing CreateTablet for tablet dfc6271b43244577a6b05d092920f236 (DEFAULT_TABLE table=table.14 [id=0a995bdfd71c4ddb9a3222d9c77c7bbb]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:01.215960 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dfc6271b43244577a6b05d092920f236. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:01.220826 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc238dfc34fc4acb9d646c80e38c01b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:01.240017 3393 tablet_bootstrap.cc:492] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:01.242542 3395 tablet_bootstrap.cc:492] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:01.242946 3393 tablet_bootstrap.cc:654] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:01.243803 3394 tablet_bootstrap.cc:492] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:01.245414 3395 tablet_bootstrap.cc:654] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:01.245847 3394 tablet_bootstrap.cc:654] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:01.246243 3393 tablet_bootstrap.cc:492] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:01.246405 3393 ts_tablet_manager.cc:1403] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251212 08:53:01.247494 3393 raft_consensus.cc:359] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:01.247736 3393 raft_consensus.cc:385] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:01.247810 3393 raft_consensus.cc:740] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:01.248064 3393 consensus_queue.cc:260] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:01.248805 3393 ts_tablet_manager.cc:1434] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20251212 08:53:01.249155 3393 tablet_bootstrap.cc:492] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:01.251583 3393 tablet_bootstrap.cc:654] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:01.255956 3395 tablet_bootstrap.cc:492] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:01.256161 3395 ts_tablet_manager.cc:1403] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.006s
I20251212 08:53:01.257278 3395 raft_consensus.cc:359] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:01.257529 3395 raft_consensus.cc:385] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:01.257594 3395 raft_consensus.cc:740] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:01.258345 3395 consensus_queue.cc:260] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:01.260462 3395 ts_tablet_manager.cc:1434] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:01.261082 3395 tablet_bootstrap.cc:492] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:01.260715 3393 tablet_bootstrap.cc:492] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:01.261314 3393 ts_tablet_manager.cc:1403] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.008s
I20251212 08:53:01.262017 3393 raft_consensus.cc:359] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:01.262176 3393 raft_consensus.cc:385] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:01.262244 3393 raft_consensus.cc:740] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:01.262491 3393 consensus_queue.cc:260] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:01.263187 3393 ts_tablet_manager.cc:1434] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:01.263329 3395 tablet_bootstrap.cc:654] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:01.263888 3394 tablet_bootstrap.cc:492] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:01.264055 3394 ts_tablet_manager.cc:1403] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.005s
I20251212 08:53:01.265877 3395 tablet_bootstrap.cc:492] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:01.266112 3395 ts_tablet_manager.cc:1403] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20251212 08:53:01.266896 3395 raft_consensus.cc:359] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:01.267063 3394 raft_consensus.cc:359] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:01.267176 3395 raft_consensus.cc:385] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:01.267270 3394 raft_consensus.cc:385] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:01.267346 3395 raft_consensus.cc:740] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:01.267362 3394 raft_consensus.cc:740] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:01.267585 3394 consensus_queue.cc:260] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:01.267620 3395 consensus_queue.cc:260] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:01.268329 3395 ts_tablet_manager.cc:1434] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:01.268842 3394 ts_tablet_manager.cc:1434] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251212 08:53:01.269150 3394 tablet_bootstrap.cc:492] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:01.272132 3394 tablet_bootstrap.cc:654] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:01.274315 3394 tablet_bootstrap.cc:492] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:01.274490 3394 ts_tablet_manager.cc:1403] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20251212 08:53:01.275357 3394 raft_consensus.cc:359] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:01.275576 3394 raft_consensus.cc:385] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:01.275636 3394 raft_consensus.cc:740] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:01.275952 3394 consensus_queue.cc:260] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:01.277624 3394 ts_tablet_manager.cc:1434] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:01.278018 3388 raft_consensus.cc:493] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:01.278131 3388 raft_consensus.cc:515] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:01.278730 3388 leader_election.cc:290] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:01.279062 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc238dfc34fc4acb9d646c80e38c01b9" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:01.279217 2885 raft_consensus.cc:2468] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:01.279232 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc238dfc34fc4acb9d646c80e38c01b9" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:01.279439 2704 raft_consensus.cc:2468] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:01.279868 2763 leader_election.cc:304] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:01.280134 3388 raft_consensus.cc:2804] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:01.280203 3388 raft_consensus.cc:493] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:01.280272 3388 raft_consensus.cc:3060] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:01.283143 3388 raft_consensus.cc:515] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:01.283694 3388 leader_election.cc:290] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:01.284034 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc238dfc34fc4acb9d646c80e38c01b9" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:01.284174 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc238dfc34fc4acb9d646c80e38c01b9" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:01.284368 2704 raft_consensus.cc:3060] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:01.284183 2885 raft_consensus.cc:3060] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:01.287096 2704 raft_consensus.cc:2468] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:01.287422 2763 leader_election.cc:304] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:01.287448 2885 raft_consensus.cc:2468] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:01.287847 3388 raft_consensus.cc:2804] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:01.288089 3388 raft_consensus.cc:697] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:01.288381 3388 consensus_queue.cc:237] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:01.291456 2585 catalog_manager.cc:5654] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:01.405398 3374 raft_consensus.cc:493] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:01.405604 3374 raft_consensus.cc:515] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:01.406270 3374 leader_election.cc:290] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:01.406654 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dfc6271b43244577a6b05d092920f236" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:01.406673 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dfc6271b43244577a6b05d092920f236" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:01.406865 2704 raft_consensus.cc:2468] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:01.406869 2807 raft_consensus.cc:2468] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:01.407272 2843 leader_election.cc:304] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:01.407490 3374 raft_consensus.cc:2804] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:01.407569 3374 raft_consensus.cc:493] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:01.407649 3374 raft_consensus.cc:3060] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:01.410157 3374 raft_consensus.cc:515] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:01.410982 3374 leader_election.cc:290] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:01.411240 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dfc6271b43244577a6b05d092920f236" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:01.411320 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "dfc6271b43244577a6b05d092920f236" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:01.411410 2704 raft_consensus.cc:3060] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:01.411444 2807 raft_consensus.cc:3060] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:01.415176 2807 raft_consensus.cc:2468] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:01.415506 2704 raft_consensus.cc:2468] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:01.415587 2845 leader_election.cc:304] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:01.416183 3374 raft_consensus.cc:2804] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:01.416340 3374 raft_consensus.cc:697] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:01.416600 3374 consensus_queue.cc:237] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:01.432605 2586 catalog_manager.cc:5654] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:01.464114 2704 raft_consensus.cc:1275] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:01.463939 2807 raft_consensus.cc:1275] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:01.464663 2702 raft_consensus.cc:1275] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:01.464829 2885 raft_consensus.cc:1275] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:01.465070 3403 consensus_queue.cc:1048] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:01.465293 3387 consensus_queue.cc:1048] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:01.465546 3403 consensus_queue.cc:1048] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:01.465643 3387 consensus_queue.cc:1048] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:01.695933 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 142ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:01.955014 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 402ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:02.214855 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 660ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:02.476738 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 921ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:02.495657 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00112e+06us
I20251212 08:53:02.497094 2585 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.15"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:02.497776 2585 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.15 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:02.507962 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 0f2534bb7ce049a19359b3c97a383cec (DEFAULT_TABLE table=table.15 [id=fb69906845cf4e69a179ceada3d320c8]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:02.508090 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 0f2534bb7ce049a19359b3c97a383cec (DEFAULT_TABLE table=table.15 [id=fb69906845cf4e69a179ceada3d320c8]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:02.508428 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f2534bb7ce049a19359b3c97a383cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:02.508435 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f2534bb7ce049a19359b3c97a383cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:02.508510 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 0f2534bb7ce049a19359b3c97a383cec (DEFAULT_TABLE table=table.15 [id=fb69906845cf4e69a179ceada3d320c8]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:02.508886 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f2534bb7ce049a19359b3c97a383cec. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:02.510293 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 38ebd93ba8814c59b510e570877bca91 (DEFAULT_TABLE table=table.15 [id=fb69906845cf4e69a179ceada3d320c8]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:02.510617 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ebd93ba8814c59b510e570877bca91. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:02.517968 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 38ebd93ba8814c59b510e570877bca91 (DEFAULT_TABLE table=table.15 [id=fb69906845cf4e69a179ceada3d320c8]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:02.518330 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 38ebd93ba8814c59b510e570877bca91 (DEFAULT_TABLE table=table.15 [id=fb69906845cf4e69a179ceada3d320c8]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:02.518813 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ebd93ba8814c59b510e570877bca91. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:02.522598 3428 tablet_bootstrap.cc:492] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:02.522900 3427 tablet_bootstrap.cc:492] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:02.518350 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 38ebd93ba8814c59b510e570877bca91. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:02.525004 3427 tablet_bootstrap.cc:654] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:02.525059 3428 tablet_bootstrap.cc:654] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:02.525991 3429 tablet_bootstrap.cc:492] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:02.527779 3429 tablet_bootstrap.cc:654] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:02.536628 3429 tablet_bootstrap.cc:492] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:02.536880 3429 ts_tablet_manager.cc:1403] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.003s
I20251212 08:53:02.537024 3427 tablet_bootstrap.cc:492] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:02.537153 3427 ts_tablet_manager.cc:1403] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.009s
I20251212 08:53:02.537797 3429 raft_consensus.cc:359] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:02.538178 3429 raft_consensus.cc:385] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:02.538288 3429 raft_consensus.cc:740] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:02.537797 3427 raft_consensus.cc:359] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:02.538395 3427 raft_consensus.cc:385] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:02.538501 3427 raft_consensus.cc:740] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:02.538491 3429 consensus_queue.cc:260] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:02.538717 3427 consensus_queue.cc:260] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:02.539114 3429 ts_tablet_manager.cc:1434] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:02.539284 3427 ts_tablet_manager.cc:1434] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:53:02.539414 3429 tablet_bootstrap.cc:492] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:02.539729 3427 tablet_bootstrap.cc:492] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:02.541162 3428 tablet_bootstrap.cc:492] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:02.541357 3428 ts_tablet_manager.cc:1403] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20251212 08:53:02.541365 3429 tablet_bootstrap.cc:654] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:02.543762 3429 tablet_bootstrap.cc:492] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:02.543967 3429 ts_tablet_manager.cc:1403] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251212 08:53:02.542279 3428 raft_consensus.cc:359] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:02.544492 3428 raft_consensus.cc:385] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:02.544579 3428 raft_consensus.cc:740] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:02.544790 3429 raft_consensus.cc:359] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:02.544940 3429 raft_consensus.cc:385] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:02.544818 3428 consensus_queue.cc:260] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:02.545038 3429 raft_consensus.cc:740] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:02.545260 3429 consensus_queue.cc:260] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:02.545581 3428 ts_tablet_manager.cc:1434] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:02.545828 3429 ts_tablet_manager.cc:1434] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:02.547459 3428 tablet_bootstrap.cc:492] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:02.549392 3428 tablet_bootstrap.cc:654] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:02.549944 3427 tablet_bootstrap.cc:654] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:02.557693 3428 tablet_bootstrap.cc:492] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:02.557952 3428 ts_tablet_manager.cc:1403] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.008s
I20251212 08:53:02.558012 3427 tablet_bootstrap.cc:492] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:02.558144 3427 ts_tablet_manager.cc:1403] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.001s
I20251212 08:53:02.559252 3428 raft_consensus.cc:359] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:02.559401 3428 raft_consensus.cc:385] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:02.559341 3427 raft_consensus.cc:359] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:02.559481 3428 raft_consensus.cc:740] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:02.559561 3427 raft_consensus.cc:385] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:02.559633 3427 raft_consensus.cc:740] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:02.559764 3428 consensus_queue.cc:260] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:02.559816 3427 consensus_queue.cc:260] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:02.560274 3428 ts_tablet_manager.cc:1434] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251212 08:53:02.560608 3427 ts_tablet_manager.cc:1434] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:02.604084 3421 raft_consensus.cc:493] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:02.604264 3421 raft_consensus.cc:515] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:02.604811 3421 leader_election.cc:290] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:02.605294 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38ebd93ba8814c59b510e570877bca91" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:02.605281 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38ebd93ba8814c59b510e570877bca91" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:02.605508 2807 raft_consensus.cc:2468] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:02.605506 2885 raft_consensus.cc:2468] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:02.605934 2667 leader_election.cc:304] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:02.606199 3421 raft_consensus.cc:2804] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:02.606271 3421 raft_consensus.cc:493] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:02.606331 3421 raft_consensus.cc:3060] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:02.609040 3351 raft_consensus.cc:493] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:02.609054 3421 raft_consensus.cc:515] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:02.609277 3351 raft_consensus.cc:515] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:02.609776 3421 leader_election.cc:290] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:02.609931 3351 leader_election.cc:290] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:02.610370 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38ebd93ba8814c59b510e570877bca91" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:02.610538 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0f2534bb7ce049a19359b3c97a383cec" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:02.610574 2806 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "38ebd93ba8814c59b510e570877bca91" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:02.610663 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0f2534bb7ce049a19359b3c97a383cec" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:02.610771 2886 raft_consensus.cc:2468] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:02.610814 2805 raft_consensus.cc:2468] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:02.610536 2885 raft_consensus.cc:3060] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:02.611230 2667 leader_election.cc:304] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:02.611480 3351 raft_consensus.cc:2804] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:02.611567 3351 raft_consensus.cc:493] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:02.611635 3351 raft_consensus.cc:3060] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:02.613667 2885 raft_consensus.cc:2468] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:02.614050 2667 leader_election.cc:304] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters: 7aad0f0d166f4050b76cd4d94dbbbc78
I20251212 08:53:02.614379 3351 raft_consensus.cc:515] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:02.614972 3351 leader_election.cc:290] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:02.615152 3351 raft_consensus.cc:2804] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:02.615262 3351 raft_consensus.cc:697] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:02.615525 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0f2534bb7ce049a19359b3c97a383cec" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:02.615525 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0f2534bb7ce049a19359b3c97a383cec" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:02.615715 2805 raft_consensus.cc:3060] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:02.615715 2886 raft_consensus.cc:3060] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:02.616469 3351 consensus_queue.cc:237] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:02.618319 2886 raft_consensus.cc:2468] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:02.618368 2805 raft_consensus.cc:2468] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:02.618769 2667 leader_election.cc:304] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:02.619153 3421 raft_consensus.cc:2804] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:02.624567 3421 raft_consensus.cc:697] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:02.624881 3421 consensus_queue.cc:237] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:02.627547 2589 catalog_manager.cc:5654] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:02.633482 2589 catalog_manager.cc:5654] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:02.651180 2885 raft_consensus.cc:1275] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:02.651542 2886 raft_consensus.cc:1275] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:02.651753 3351 consensus_queue.cc:1048] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:02.652356 2807 raft_consensus.cc:3060] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:02.652668 2805 raft_consensus.cc:1275] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:02.652666 3421 consensus_queue.cc:1048] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:02.653204 3351 consensus_queue.cc:1048] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:02.655576 2807 raft_consensus.cc:1275] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:02.656038 3421 consensus_queue.cc:1048] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:02.935354 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 209ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:03.195276 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 467ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:03.455263 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 727ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:03.669600 2738 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.1:35409, user_credentials={real_user=slave}} blocked reactor thread for 1.00115e+06us
I20251212 08:53:03.671041 2589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.16"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:03.671749 2589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.16 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:03.684929 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 86f83858fbcd4ed18d56f089d34d7a98 (DEFAULT_TABLE table=table.16 [id=ea87c48d63c8487b9ebd45ec62ba1f94]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:03.685066 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 86f83858fbcd4ed18d56f089d34d7a98 (DEFAULT_TABLE table=table.16 [id=ea87c48d63c8487b9ebd45ec62ba1f94]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:03.685391 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86f83858fbcd4ed18d56f089d34d7a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:03.685391 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86f83858fbcd4ed18d56f089d34d7a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:03.685485 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 12a808ed739f4d159faa084914e966dd (DEFAULT_TABLE table=table.16 [id=ea87c48d63c8487b9ebd45ec62ba1f94]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:03.685858 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a808ed739f4d159faa084914e966dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:03.687908 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 12a808ed739f4d159faa084914e966dd (DEFAULT_TABLE table=table.16 [id=ea87c48d63c8487b9ebd45ec62ba1f94]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:03.688277 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a808ed739f4d159faa084914e966dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:03.690747 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 86f83858fbcd4ed18d56f089d34d7a98 (DEFAULT_TABLE table=table.16 [id=ea87c48d63c8487b9ebd45ec62ba1f94]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:03.691110 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 86f83858fbcd4ed18d56f089d34d7a98. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:03.695199 3459 tablet_bootstrap.cc:492] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:03.697047 3459 tablet_bootstrap.cc:654] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:03.698562 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 12a808ed739f4d159faa084914e966dd (DEFAULT_TABLE table=table.16 [id=ea87c48d63c8487b9ebd45ec62ba1f94]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:03.699010 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 12a808ed739f4d159faa084914e966dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:03.714710 3460 tablet_bootstrap.cc:492] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:03.717162 3460 tablet_bootstrap.cc:654] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:03.717756 3459 tablet_bootstrap.cc:492] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:03.717957 3459 ts_tablet_manager.cc:1403] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.023s user 0.003s sys 0.003s
I20251212 08:53:03.719357 3459 raft_consensus.cc:359] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:03.719718 3459 raft_consensus.cc:385] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:03.719861 3459 raft_consensus.cc:740] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:03.720240 3461 tablet_bootstrap.cc:492] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:03.720829 3459 consensus_queue.cc:260] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:03.722048 3459 ts_tablet_manager.cc:1434] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20251212 08:53:03.722381 3459 tablet_bootstrap.cc:492] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:03.723619 3461 tablet_bootstrap.cc:654] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:03.725131 3459 tablet_bootstrap.cc:654] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:03.727764 3460 tablet_bootstrap.cc:492] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:03.727929 3460 ts_tablet_manager.cc:1403] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20251212 08:53:03.729014 3460 raft_consensus.cc:359] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:03.729353 3460 raft_consensus.cc:385] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:03.729480 3460 raft_consensus.cc:740] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:03.729828 3460 consensus_queue.cc:260] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:03.730727 3460 ts_tablet_manager.cc:1434] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20251212 08:53:03.731102 3460 tablet_bootstrap.cc:492] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:03.734243 3459 tablet_bootstrap.cc:492] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:03.734462 3459 ts_tablet_manager.cc:1403] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20251212 08:53:03.735168 3459 raft_consensus.cc:359] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:03.735467 3459 raft_consensus.cc:385] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:03.735589 3459 raft_consensus.cc:740] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:03.735929 3459 consensus_queue.cc:260] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:03.739501 3459 ts_tablet_manager.cc:1434] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20251212 08:53:03.735657 3460 tablet_bootstrap.cc:654] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:03.744812 3460 tablet_bootstrap.cc:492] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:03.745038 3460 ts_tablet_manager.cc:1403] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.005s
I20251212 08:53:03.745749 3461 tablet_bootstrap.cc:492] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:03.745939 3461 ts_tablet_manager.cc:1403] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.026s user 0.003s sys 0.011s
I20251212 08:53:03.746419 3460 raft_consensus.cc:359] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:03.746698 3460 raft_consensus.cc:385] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:03.746824 3460 raft_consensus.cc:740] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:03.746977 3461 raft_consensus.cc:359] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:03.747197 3461 raft_consensus.cc:385] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:03.747145 3460 consensus_queue.cc:260] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:03.747300 3461 raft_consensus.cc:740] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:03.747629 3461 consensus_queue.cc:260] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:03.747860 3460 ts_tablet_manager.cc:1434] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251212 08:53:03.749387 3461 ts_tablet_manager.cc:1434] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251212 08:53:03.749640 3461 tablet_bootstrap.cc:492] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:03.751693 3461 tablet_bootstrap.cc:654] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:03.759567 3461 tablet_bootstrap.cc:492] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:03.759709 3461 ts_tablet_manager.cc:1403] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20251212 08:53:03.760269 3461 raft_consensus.cc:359] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:03.760442 3461 raft_consensus.cc:385] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:03.760476 3461 raft_consensus.cc:740] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:03.760654 3461 consensus_queue.cc:260] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:03.761168 3461 ts_tablet_manager.cc:1434] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:03.886857 3351 raft_consensus.cc:493] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:03.887018 3351 raft_consensus.cc:515] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:03.887655 3351 leader_election.cc:290] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:03.888171 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86f83858fbcd4ed18d56f089d34d7a98" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:03.888164 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86f83858fbcd4ed18d56f089d34d7a98" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:03.888443 2807 raft_consensus.cc:2468] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:03.888443 2885 raft_consensus.cc:2468] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:03.888914 2667 leader_election.cc:304] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:03.889194 3351 raft_consensus.cc:2804] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:03.889276 3351 raft_consensus.cc:493] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:03.889336 3351 raft_consensus.cc:3060] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:03.908618 3351 raft_consensus.cc:515] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:03.909148 3351 leader_election.cc:290] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:03.909583 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86f83858fbcd4ed18d56f089d34d7a98" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:03.909749 2807 raft_consensus.cc:3060] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:03.909693 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "86f83858fbcd4ed18d56f089d34d7a98" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:03.909950 2885 raft_consensus.cc:3060] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:03.911764 2885 raft_consensus.cc:2468] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:03.911772 2807 raft_consensus.cc:2468] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:03.912137 2667 leader_election.cc:304] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:03.912417 3351 raft_consensus.cc:2804] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:03.912827 3351 raft_consensus.cc:697] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:03.913080 3351 consensus_queue.cc:237] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:03.916447 2585 catalog_manager.cc:5654] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:03.921397 3351 raft_consensus.cc:493] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:03.921530 3351 raft_consensus.cc:515] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:03.921968 3351 leader_election.cc:290] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:03.922331 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12a808ed739f4d159faa084914e966dd" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:03.922421 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12a808ed739f4d159faa084914e966dd" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:03.922469 2807 raft_consensus.cc:2468] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:03.922612 2885 raft_consensus.cc:2468] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:03.922857 2667 leader_election.cc:304] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:03.923044 3351 raft_consensus.cc:2804] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:03.923148 3351 raft_consensus.cc:493] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:03.923226 3351 raft_consensus.cc:3060] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:03.925240 3351 raft_consensus.cc:515] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:03.925786 3351 leader_election.cc:290] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:03.926214 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12a808ed739f4d159faa084914e966dd" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:03.926365 2885 raft_consensus.cc:3060] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:03.926435 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "12a808ed739f4d159faa084914e966dd" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:03.926548 2807 raft_consensus.cc:3060] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:03.929104 2885 raft_consensus.cc:2468] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:03.929569 2667 leader_election.cc:304] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:03.929104 2807 raft_consensus.cc:2468] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:03.929788 3351 raft_consensus.cc:2804] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:03.929908 3351 raft_consensus.cc:697] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:03.930249 3351 consensus_queue.cc:237] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:03.934603 2586 catalog_manager.cc:5654] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:03.994539 2885 raft_consensus.cc:1275] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:03.995507 2807 raft_consensus.cc:1275] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:03.995674 2805 raft_consensus.cc:1275] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:03.995450 3469 consensus_queue.cc:1048] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:03.995985 2885 raft_consensus.cc:1275] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:03.996352 3469 consensus_queue.cc:1048] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:03.996536 3351 consensus_queue.cc:1048] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:03.996908 3469 consensus_queue.cc:1048] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:04.027236 3478 mvcc.cc:204] Tried to move back new op lower bound from 7231609176036835328 to 7231609175710576640. Current Snapshot: MvccSnapshot[applied={T|T < 7231609176036835328 or (T in {7231609176036835328})}]
W20251212 08:53:04.216157 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 134ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:04.476794 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 394ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:04.737814 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 654ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:04.997671 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 915ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:05.023520 2738 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.1:35409, user_credentials={real_user=slave}} blocked reactor thread for 1.00116e+06us
I20251212 08:53:05.025363 2586 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.17"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:05.026109 2586 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.17 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:05.037441 2795 tablet_service.cc:1505] Processing CreateTablet for tablet d4841a0029994cc3a2d02e5889d34958 (DEFAULT_TABLE table=table.17 [id=1352aefa91514801a0c20cc5befcd4d0]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:05.037918 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4841a0029994cc3a2d02e5889d34958. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:05.037912 2694 tablet_service.cc:1505] Processing CreateTablet for tablet d4841a0029994cc3a2d02e5889d34958 (DEFAULT_TABLE table=table.17 [id=1352aefa91514801a0c20cc5befcd4d0]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:05.038241 2876 tablet_service.cc:1505] Processing CreateTablet for tablet d4841a0029994cc3a2d02e5889d34958 (DEFAULT_TABLE table=table.17 [id=1352aefa91514801a0c20cc5befcd4d0]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:05.038439 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4841a0029994cc3a2d02e5889d34958. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:05.038564 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4841a0029994cc3a2d02e5889d34958. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:05.045487 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 9bbebd39db8945459705e59f8670eb3f (DEFAULT_TABLE table=table.17 [id=1352aefa91514801a0c20cc5befcd4d0]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:05.046017 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bbebd39db8945459705e59f8670eb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:05.046511 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 9bbebd39db8945459705e59f8670eb3f (DEFAULT_TABLE table=table.17 [id=1352aefa91514801a0c20cc5befcd4d0]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:05.048077 3496 tablet_bootstrap.cc:492] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:05.048408 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bbebd39db8945459705e59f8670eb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:05.057118 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 9bbebd39db8945459705e59f8670eb3f (DEFAULT_TABLE table=table.17 [id=1352aefa91514801a0c20cc5befcd4d0]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:05.057647 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9bbebd39db8945459705e59f8670eb3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:05.058291 3497 tablet_bootstrap.cc:492] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:05.060324 3497 tablet_bootstrap.cc:654] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:05.060343 3495 tablet_bootstrap.cc:492] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:05.063031 3495 tablet_bootstrap.cc:654] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:05.065429 3496 tablet_bootstrap.cc:654] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:05.068416 3496 tablet_bootstrap.cc:492] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:05.068689 3496 ts_tablet_manager.cc:1403] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.002s
I20251212 08:53:05.070420 3495 tablet_bootstrap.cc:492] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:05.070354 3496 raft_consensus.cc:359] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:05.070675 3496 raft_consensus.cc:385] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:05.070777 3495 ts_tablet_manager.cc:1403] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20251212 08:53:05.070856 3496 raft_consensus.cc:740] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:05.071259 3496 consensus_queue.cc:260] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:05.071661 3495 raft_consensus.cc:359] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:05.071818 3495 raft_consensus.cc:385] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:05.071914 3495 raft_consensus.cc:740] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:05.072132 3495 consensus_queue.cc:260] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:05.072732 3495 ts_tablet_manager.cc:1434] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:05.072855 3497 tablet_bootstrap.cc:492] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:05.072985 3497 ts_tablet_manager.cc:1403] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20251212 08:53:05.073300 3496 ts_tablet_manager.cc:1434] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20251212 08:53:05.073642 3496 tablet_bootstrap.cc:492] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:05.073839 3497 raft_consensus.cc:359] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:05.074052 3497 raft_consensus.cc:385] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:05.074190 3497 raft_consensus.cc:740] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:05.074456 3497 consensus_queue.cc:260] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:05.075064 3497 ts_tablet_manager.cc:1434] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:05.075457 3497 tablet_bootstrap.cc:492] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:05.076561 3495 tablet_bootstrap.cc:492] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:05.076946 3496 tablet_bootstrap.cc:654] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:05.077517 3497 tablet_bootstrap.cc:654] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:05.080240 3497 tablet_bootstrap.cc:492] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:05.080391 3495 tablet_bootstrap.cc:654] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:05.080469 3497 ts_tablet_manager.cc:1403] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251212 08:53:05.081907 3497 raft_consensus.cc:359] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.082206 3497 raft_consensus.cc:385] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:05.082319 3497 raft_consensus.cc:740] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:05.082587 3497 consensus_queue.cc:260] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.082783 3496 tablet_bootstrap.cc:492] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:05.083209 3497 ts_tablet_manager.cc:1434] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20251212 08:53:05.083269 3495 tablet_bootstrap.cc:492] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:05.083423 3495 ts_tablet_manager.cc:1403] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20251212 08:53:05.084165 3495 raft_consensus.cc:359] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.084316 3496 ts_tablet_manager.cc:1403] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.001s
I20251212 08:53:05.084385 3495 raft_consensus.cc:385] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:05.084517 3495 raft_consensus.cc:740] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:05.084839 3495 consensus_queue.cc:260] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.085052 3496 raft_consensus.cc:359] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.085239 3496 raft_consensus.cc:385] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:05.085340 3496 raft_consensus.cc:740] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:05.085433 3495 ts_tablet_manager.cc:1434] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:05.085574 3496 consensus_queue.cc:260] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.087242 3496 ts_tablet_manager.cc:1434] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20251212 08:53:05.115805 3504 raft_consensus.cc:493] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:05.116047 3504 raft_consensus.cc:515] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:05.117144 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d4841a0029994cc3a2d02e5889d34958" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:05.117168 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d4841a0029994cc3a2d02e5889d34958" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:05.117377 2807 raft_consensus.cc:2468] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:05.117435 2704 raft_consensus.cc:2468] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:05.117782 2845 leader_election.cc:304] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:05.117847 3504 leader_election.cc:290] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:05.118067 3374 raft_consensus.cc:2804] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:05.118175 3374 raft_consensus.cc:493] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:05.118280 3374 raft_consensus.cc:3060] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:05.120744 3374 raft_consensus.cc:515] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:05.121281 3374 leader_election.cc:290] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:05.121712 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d4841a0029994cc3a2d02e5889d34958" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:05.121712 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d4841a0029994cc3a2d02e5889d34958" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:05.121881 2704 raft_consensus.cc:3060] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:05.121942 2807 raft_consensus.cc:3060] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:05.124557 2704 raft_consensus.cc:2468] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:05.124586 2807 raft_consensus.cc:2468] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:05.125006 2845 leader_election.cc:304] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:05.125362 3374 raft_consensus.cc:2804] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:05.125530 3374 raft_consensus.cc:697] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:05.125797 3374 consensus_queue.cc:237] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:05.129731 2585 catalog_manager.cc:5654] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:05.452198 3374 raft_consensus.cc:493] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:05.452391 3374 raft_consensus.cc:515] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.453099 3351 raft_consensus.cc:493] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:05.453282 3351 raft_consensus.cc:515] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.453536 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9bbebd39db8945459705e59f8670eb3f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:05.453702 2807 raft_consensus.cc:2468] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:05.454046 3351 leader_election.cc:290] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:05.454064 2845 leader_election.cc:304] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:05.453102 3374 leader_election.cc:290] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:05.454710 3504 raft_consensus.cc:2804] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:05.454773 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9bbebd39db8945459705e59f8670eb3f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:05.454875 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9bbebd39db8945459705e59f8670eb3f" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:05.454933 2704 raft_consensus.cc:2468] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:05.454787 3504 raft_consensus.cc:493] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:05.455189 3504 raft_consensus.cc:3060] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:05.455333 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9bbebd39db8945459705e59f8670eb3f" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:05.455471 2807 raft_consensus.cc:2468] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:05.455803 2667 leader_election.cc:304] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:05.456714 3351 raft_consensus.cc:2804] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:05.456815 3351 raft_consensus.cc:493] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:05.456882 3351 raft_consensus.cc:3060] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:05.457899 3504 raft_consensus.cc:515] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.458194 2886 raft_consensus.cc:2393] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7cab108f9abb4e1c973338419932d96c in current term 1: Already voted for candidate 2d41c990aa9f453e96ec315fab508132 in this term.
I20251212 08:53:05.458609 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9bbebd39db8945459705e59f8670eb3f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:05.458750 2807 raft_consensus.cc:3060] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:05.458738 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9bbebd39db8945459705e59f8670eb3f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:05.459087 3351 raft_consensus.cc:515] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.459391 2704 raft_consensus.cc:2393] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 2d41c990aa9f453e96ec315fab508132 in current term 1: Already voted for candidate 7cab108f9abb4e1c973338419932d96c in this term.
I20251212 08:53:05.459599 3351 leader_election.cc:290] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:05.460141 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9bbebd39db8945459705e59f8670eb3f" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:05.460337 2886 raft_consensus.cc:2393] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7cab108f9abb4e1c973338419932d96c in current term 1: Already voted for candidate 2d41c990aa9f453e96ec315fab508132 in this term.
I20251212 08:53:05.460551 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9bbebd39db8945459705e59f8670eb3f" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:05.461025 2667 leader_election.cc:304] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7cab108f9abb4e1c973338419932d96c; no voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78
I20251212 08:53:05.461293 3351 raft_consensus.cc:2749] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251212 08:53:05.461426 3504 leader_election.cc:290] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:05.461316 2807 raft_consensus.cc:2468] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:05.461789 2845 leader_election.cc:304] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters: 7cab108f9abb4e1c973338419932d96c
I20251212 08:53:05.462034 3504 raft_consensus.cc:2804] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:05.462214 3504 raft_consensus.cc:697] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:05.462472 3504 consensus_queue.cc:237] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:05.465536 2589 catalog_manager.cc:5654] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:05.509711 2704 raft_consensus.cc:1275] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:05.510047 2807 raft_consensus.cc:1275] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:05.511044 3374 consensus_queue.cc:1048] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:05.512434 2704 raft_consensus.cc:1275] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:05.512651 3511 consensus_queue.cc:1048] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:05.513796 3511 consensus_queue.cc:1048] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:05.531985 2807 raft_consensus.cc:1275] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:05.533861 3374 consensus_queue.cc:1048] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:05.861958 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 269ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:06.125474 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 532ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:06.391048 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 795ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:06.531612 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.3:33005, user_credentials={real_user=slave}} blocked reactor thread for 1.001e+06us
I20251212 08:53:06.533080 2585 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.18"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:06.534186 2585 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.18 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:06.543233 2793 tablet_service.cc:1505] Processing CreateTablet for tablet db9329c939f846f48bd173a350bef36d (DEFAULT_TABLE table=table.18 [id=7920294993674d94834cd5df0798dac1]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:06.543663 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9329c939f846f48bd173a350bef36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:06.543663 2873 tablet_service.cc:1505] Processing CreateTablet for tablet db9329c939f846f48bd173a350bef36d (DEFAULT_TABLE table=table.18 [id=7920294993674d94834cd5df0798dac1]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:06.544070 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9329c939f846f48bd173a350bef36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:06.544515 2694 tablet_service.cc:1505] Processing CreateTablet for tablet bcd3ea4b4e7341b59b92ce737d89af33 (DEFAULT_TABLE table=table.18 [id=7920294993674d94834cd5df0798dac1]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:06.544726 2695 tablet_service.cc:1505] Processing CreateTablet for tablet db9329c939f846f48bd173a350bef36d (DEFAULT_TABLE table=table.18 [id=7920294993674d94834cd5df0798dac1]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:06.544893 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd3ea4b4e7341b59b92ce737d89af33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:06.545231 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet db9329c939f846f48bd173a350bef36d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:06.548637 2876 tablet_service.cc:1505] Processing CreateTablet for tablet bcd3ea4b4e7341b59b92ce737d89af33 (DEFAULT_TABLE table=table.18 [id=7920294993674d94834cd5df0798dac1]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:06.549019 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd3ea4b4e7341b59b92ce737d89af33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:06.553323 2795 tablet_service.cc:1505] Processing CreateTablet for tablet bcd3ea4b4e7341b59b92ce737d89af33 (DEFAULT_TABLE table=table.18 [id=7920294993674d94834cd5df0798dac1]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:06.553691 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcd3ea4b4e7341b59b92ce737d89af33. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:06.563664 3530 tablet_bootstrap.cc:492] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:06.568427 3530 tablet_bootstrap.cc:654] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:06.574566 3531 tablet_bootstrap.cc:492] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:06.577175 3531 tablet_bootstrap.cc:654] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:06.579314 3529 tablet_bootstrap.cc:492] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:06.580163 3530 tablet_bootstrap.cc:492] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:06.580389 3530 ts_tablet_manager.cc:1403] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.004s
I20251212 08:53:06.581355 3530 raft_consensus.cc:359] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:06.581593 3530 raft_consensus.cc:385] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:06.581589 3529 tablet_bootstrap.cc:654] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:06.582337 3531 tablet_bootstrap.cc:492] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:06.582762 3531 ts_tablet_manager.cc:1403] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20251212 08:53:06.583616 3531 raft_consensus.cc:359] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:06.583868 3531 raft_consensus.cc:385] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:06.583964 3531 raft_consensus.cc:740] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:06.584249 3531 consensus_queue.cc:260] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:06.586344 3531 ts_tablet_manager.cc:1434] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:06.586716 3531 tablet_bootstrap.cc:492] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:06.588840 3531 tablet_bootstrap.cc:654] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:06.582420 3530 raft_consensus.cc:740] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:06.590086 3530 consensus_queue.cc:260] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:06.590682 3530 ts_tablet_manager.cc:1434] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.010s user 0.002s sys 0.000s
I20251212 08:53:06.592550 3530 tablet_bootstrap.cc:492] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:06.593151 3529 tablet_bootstrap.cc:492] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:06.593299 3529 ts_tablet_manager.cc:1403] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.010s
I20251212 08:53:06.594069 3529 raft_consensus.cc:359] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:06.594247 3529 raft_consensus.cc:385] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:06.594339 3529 raft_consensus.cc:740] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:06.594554 3529 consensus_queue.cc:260] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:06.595083 3529 ts_tablet_manager.cc:1434] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:53:06.595342 3529 tablet_bootstrap.cc:492] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:06.595423 3530 tablet_bootstrap.cc:654] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:06.597404 3531 tablet_bootstrap.cc:492] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:06.597617 3531 ts_tablet_manager.cc:1403] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20251212 08:53:06.597711 3529 tablet_bootstrap.cc:654] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:06.598510 3531 raft_consensus.cc:359] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:06.598786 3531 raft_consensus.cc:385] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:06.598884 3531 raft_consensus.cc:740] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:06.599107 3531 consensus_queue.cc:260] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:06.599957 3531 ts_tablet_manager.cc:1434] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:06.601120 3530 tablet_bootstrap.cc:492] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:06.601326 3530 ts_tablet_manager.cc:1403] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20251212 08:53:06.602234 3530 raft_consensus.cc:359] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:06.602517 3530 raft_consensus.cc:385] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:06.602641 3530 raft_consensus.cc:740] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:06.602975 3530 consensus_queue.cc:260] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:06.603704 3530 ts_tablet_manager.cc:1434] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:06.607326 3529 tablet_bootstrap.cc:492] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:06.607585 3529 ts_tablet_manager.cc:1403] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.002s
I20251212 08:53:06.608489 3529 raft_consensus.cc:359] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:06.608707 3529 raft_consensus.cc:385] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:06.608786 3529 raft_consensus.cc:740] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:06.609012 3529 consensus_queue.cc:260] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:06.609606 3529 ts_tablet_manager.cc:1434] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:06.633205 3494 raft_consensus.cc:493] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:06.633376 3494 raft_consensus.cc:515] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:06.634117 3494 leader_election.cc:290] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:06.634402 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "db9329c939f846f48bd173a350bef36d" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:06.634493 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "db9329c939f846f48bd173a350bef36d" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:06.634585 2886 raft_consensus.cc:2468] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:06.634678 2705 raft_consensus.cc:2468] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:06.634925 2765 leader_election.cc:304] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:06.635177 3494 raft_consensus.cc:2804] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:06.635254 3494 raft_consensus.cc:493] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:06.635332 3494 raft_consensus.cc:3060] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:06.637403 3494 raft_consensus.cc:515] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:06.638036 3494 leader_election.cc:290] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:06.638384 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "db9329c939f846f48bd173a350bef36d" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:06.638386 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "db9329c939f846f48bd173a350bef36d" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:06.638550 2886 raft_consensus.cc:3060] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:06.638602 2705 raft_consensus.cc:3060] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:06.640676 2886 raft_consensus.cc:2468] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:06.641029 2765 leader_election.cc:304] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:06.641263 3494 raft_consensus.cc:2804] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:06.641222 2705 raft_consensus.cc:2468] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:06.641510 3494 raft_consensus.cc:697] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:06.641804 3494 consensus_queue.cc:237] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:06.645080 2589 catalog_manager.cc:5654] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:06.675979 3494 raft_consensus.cc:493] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:06.676157 3494 raft_consensus.cc:515] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:06.676800 3494 leader_election.cc:290] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:06.677090 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bcd3ea4b4e7341b59b92ce737d89af33" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:06.677183 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bcd3ea4b4e7341b59b92ce737d89af33" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:06.677258 2705 raft_consensus.cc:2468] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:06.677353 2886 raft_consensus.cc:2468] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:06.677647 2763 leader_election.cc:304] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:06.677903 3494 raft_consensus.cc:2804] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:06.677982 3494 raft_consensus.cc:493] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:06.678048 3494 raft_consensus.cc:3060] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:06.680174 3494 raft_consensus.cc:515] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:06.680711 3494 leader_election.cc:290] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:06.681011 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bcd3ea4b4e7341b59b92ce737d89af33" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:06.681095 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bcd3ea4b4e7341b59b92ce737d89af33" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:06.681185 2705 raft_consensus.cc:3060] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:06.681252 2886 raft_consensus.cc:3060] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:06.684319 2705 raft_consensus.cc:2468] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:06.684924 2763 leader_election.cc:304] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:06.685220 3494 raft_consensus.cc:2804] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:06.685501 3494 raft_consensus.cc:697] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:06.685760 3494 consensus_queue.cc:237] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:06.686146 2886 raft_consensus.cc:2468] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:06.689997 2589 catalog_manager.cc:5654] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:06.716405 2705 raft_consensus.cc:1275] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:06.716996 3542 consensus_queue.cc:1048] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:06.717186 2705 raft_consensus.cc:1275] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:06.717217 2887 raft_consensus.cc:1275] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:06.717186 2886 raft_consensus.cc:1275] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:06.718197 3493 consensus_queue.cc:1048] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:06.718575 3493 consensus_queue.cc:1048] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:06.718206 3494 consensus_queue.cc:1048] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20251212 08:53:07.052578 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 251ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:07.314399 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 511ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:07.574752 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 773ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:07.742841 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00088e+06us
I20251212 08:53:07.744256 2589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.19"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:07.745208 2589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.19 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:07.756440 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 337002230a3949a1af139ac4ac693f81 (DEFAULT_TABLE table=table.19 [id=8b01f87e0d984a2fa786d46900bbcd3e]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:07.756611 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 337002230a3949a1af139ac4ac693f81 (DEFAULT_TABLE table=table.19 [id=8b01f87e0d984a2fa786d46900bbcd3e]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:07.756893 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 337002230a3949a1af139ac4ac693f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:07.757434 2695 tablet_service.cc:1505] Processing CreateTablet for tablet c0c0068af3ec4fa1b34b33e05ec9da93 (DEFAULT_TABLE table=table.19 [id=8b01f87e0d984a2fa786d46900bbcd3e]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:07.757481 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 337002230a3949a1af139ac4ac693f81 (DEFAULT_TABLE table=table.19 [id=8b01f87e0d984a2fa786d46900bbcd3e]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:07.757879 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 337002230a3949a1af139ac4ac693f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:07.756893 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 337002230a3949a1af139ac4ac693f81. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:07.758373 2795 tablet_service.cc:1505] Processing CreateTablet for tablet c0c0068af3ec4fa1b34b33e05ec9da93 (DEFAULT_TABLE table=table.19 [id=8b01f87e0d984a2fa786d46900bbcd3e]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:07.758711 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c0068af3ec4fa1b34b33e05ec9da93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:07.759727 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c0068af3ec4fa1b34b33e05ec9da93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:07.759990 2876 tablet_service.cc:1505] Processing CreateTablet for tablet c0c0068af3ec4fa1b34b33e05ec9da93 (DEFAULT_TABLE table=table.19 [id=8b01f87e0d984a2fa786d46900bbcd3e]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:07.760334 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0c0068af3ec4fa1b34b33e05ec9da93. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:07.776986 3563 tablet_bootstrap.cc:492] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:07.779188 3561 tablet_bootstrap.cc:492] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:07.780793 3562 tablet_bootstrap.cc:492] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:07.781369 3563 tablet_bootstrap.cc:654] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:07.781452 3561 tablet_bootstrap.cc:654] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:07.782982 3562 tablet_bootstrap.cc:654] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:07.787998 3562 tablet_bootstrap.cc:492] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:07.788224 3562 ts_tablet_manager.cc:1403] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20251212 08:53:07.789320 3562 raft_consensus.cc:359] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:07.789558 3561 tablet_bootstrap.cc:492] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:07.789644 3563 tablet_bootstrap.cc:492] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:07.789805 3561 ts_tablet_manager.cc:1403] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20251212 08:53:07.789916 3562 raft_consensus.cc:385] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:07.789997 3562 raft_consensus.cc:740] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:07.789789 3563 ts_tablet_manager.cc:1403] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20251212 08:53:07.790324 3562 consensus_queue.cc:260] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:07.790855 3562 ts_tablet_manager.cc:1434] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:53:07.791332 3562 tablet_bootstrap.cc:492] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:07.791569 3561 raft_consensus.cc:359] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:07.791786 3561 raft_consensus.cc:385] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:07.791852 3563 raft_consensus.cc:359] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:07.792075 3563 raft_consensus.cc:385] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:07.792155 3563 raft_consensus.cc:740] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:07.792446 3563 consensus_queue.cc:260] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:07.793133 3563 ts_tablet_manager.cc:1434] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:53:07.791875 3561 raft_consensus.cc:740] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:07.793455 3563 tablet_bootstrap.cc:492] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:07.793511 3562 tablet_bootstrap.cc:654] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:07.793493 3561 consensus_queue.cc:260] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:07.795480 3563 tablet_bootstrap.cc:654] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:07.798471 3561 ts_tablet_manager.cc:1434] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20251212 08:53:07.799890 3561 tablet_bootstrap.cc:492] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:07.802394 3562 tablet_bootstrap.cc:492] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:07.802664 3562 ts_tablet_manager.cc:1403] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20251212 08:53:07.802779 3561 tablet_bootstrap.cc:654] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:07.803791 3562 raft_consensus.cc:359] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:07.804034 3562 raft_consensus.cc:385] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:07.804153 3562 raft_consensus.cc:740] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:07.804476 3561 tablet_bootstrap.cc:492] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:07.804466 3562 consensus_queue.cc:260] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:07.804644 3561 ts_tablet_manager.cc:1403] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251212 08:53:07.805141 3562 ts_tablet_manager.cc:1434] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:07.805404 3561 raft_consensus.cc:359] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:07.805629 3561 raft_consensus.cc:385] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:07.805696 3561 raft_consensus.cc:740] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:07.805946 3561 consensus_queue.cc:260] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:07.806492 3563 tablet_bootstrap.cc:492] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:07.806612 3563 ts_tablet_manager.cc:1403] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.007s
I20251212 08:53:07.807119 3563 raft_consensus.cc:359] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:07.807235 3563 raft_consensus.cc:385] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:07.807310 3563 raft_consensus.cc:740] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:07.807482 3563 consensus_queue.cc:260] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:07.807591 3561 ts_tablet_manager.cc:1434] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:53:07.809384 3563 ts_tablet_manager.cc:1434] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:53:07.825083 3548 raft_consensus.cc:493] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:07.825273 3548 raft_consensus.cc:515] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:07.825971 3548 leader_election.cc:290] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:07.826941 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0c0068af3ec4fa1b34b33e05ec9da93" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:07.827118 2807 raft_consensus.cc:2468] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:07.827514 2845 leader_election.cc:304] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:07.827812 3548 raft_consensus.cc:2804] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:07.827913 3548 raft_consensus.cc:493] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:07.827991 3548 raft_consensus.cc:3060] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:07.827525 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0c0068af3ec4fa1b34b33e05ec9da93" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:07.828634 2705 raft_consensus.cc:2468] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:07.830117 3548 raft_consensus.cc:515] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:07.830595 3548 leader_election.cc:290] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:07.830988 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0c0068af3ec4fa1b34b33e05ec9da93" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:07.831009 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0c0068af3ec4fa1b34b33e05ec9da93" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:07.831159 2705 raft_consensus.cc:3060] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:07.831173 2807 raft_consensus.cc:3060] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:07.833689 2705 raft_consensus.cc:2468] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:07.833846 2807 raft_consensus.cc:2468] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:07.834203 2843 leader_election.cc:304] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:07.834539 3548 raft_consensus.cc:2804] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:07.834658 3548 raft_consensus.cc:697] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:07.834909 3548 consensus_queue.cc:237] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:07.838111 2589 catalog_manager.cc:5654] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:08.059305 3543 raft_consensus.cc:493] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:08.059497 3543 raft_consensus.cc:515] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:08.060145 3543 leader_election.cc:290] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:08.061074 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "337002230a3949a1af139ac4ac693f81" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:08.061250 2705 raft_consensus.cc:2468] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:08.061404 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "337002230a3949a1af139ac4ac693f81" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:08.061553 2885 raft_consensus.cc:2468] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:08.061730 2763 leader_election.cc:304] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:08.062038 3543 raft_consensus.cc:2804] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:08.062115 3543 raft_consensus.cc:493] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:08.062181 3543 raft_consensus.cc:3060] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:08.064256 3543 raft_consensus.cc:515] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:08.064776 3543 leader_election.cc:290] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:08.065162 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "337002230a3949a1af139ac4ac693f81" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:08.065196 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "337002230a3949a1af139ac4ac693f81" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:08.065321 2885 raft_consensus.cc:3060] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:08.065303 2705 raft_consensus.cc:3060] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:08.067317 2885 raft_consensus.cc:2468] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:08.067425 2705 raft_consensus.cc:2468] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:08.067687 2765 leader_election.cc:304] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:08.067929 3543 raft_consensus.cc:2804] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:08.068104 3543 raft_consensus.cc:697] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:08.068401 3543 consensus_queue.cc:237] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:08.072190 2587 catalog_manager.cc:5654] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:08.138968 2807 raft_consensus.cc:1275] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:08.139165 2705 raft_consensus.cc:1275] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:08.139583 3548 consensus_queue.cc:1048] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:08.139657 3493 consensus_queue.cc:1048] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:08.140178 2885 raft_consensus.cc:1275] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:08.140803 3493 consensus_queue.cc:1048] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:08.142251 2704 raft_consensus.cc:1275] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:08.144090 3548 consensus_queue.cc:1048] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:08.438074 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 216ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:08.702543 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 478ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:08.965834 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 743ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:09.160977 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00101e+06us
I20251212 08:53:09.162477 2585 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.20"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:09.163285 2585 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.20 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:09.174599 2795 tablet_service.cc:1505] Processing CreateTablet for tablet bbe34e1fdbf24c1c9c1edc1e14bfa88d (DEFAULT_TABLE table=table.20 [id=68e0d80df339448b85ac663f519eed12]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:09.174644 2873 tablet_service.cc:1505] Processing CreateTablet for tablet bbe34e1fdbf24c1c9c1edc1e14bfa88d (DEFAULT_TABLE table=table.20 [id=68e0d80df339448b85ac663f519eed12]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:09.175087 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbe34e1fdbf24c1c9c1edc1e14bfa88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:09.175048 2694 tablet_service.cc:1505] Processing CreateTablet for tablet bbe34e1fdbf24c1c9c1edc1e14bfa88d (DEFAULT_TABLE table=table.20 [id=68e0d80df339448b85ac663f519eed12]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:09.175089 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbe34e1fdbf24c1c9c1edc1e14bfa88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:09.175352 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbe34e1fdbf24c1c9c1edc1e14bfa88d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:09.179517 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 7d16ad80c41b41e2accecd66ca20e04a (DEFAULT_TABLE table=table.20 [id=68e0d80df339448b85ac663f519eed12]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:09.179906 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d16ad80c41b41e2accecd66ca20e04a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:09.179817 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 7d16ad80c41b41e2accecd66ca20e04a (DEFAULT_TABLE table=table.20 [id=68e0d80df339448b85ac663f519eed12]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:09.180243 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d16ad80c41b41e2accecd66ca20e04a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:09.180590 3596 tablet_bootstrap.cc:492] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:09.179517 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 7d16ad80c41b41e2accecd66ca20e04a (DEFAULT_TABLE table=table.20 [id=68e0d80df339448b85ac663f519eed12]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:09.185416 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7d16ad80c41b41e2accecd66ca20e04a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:09.187325 3596 tablet_bootstrap.cc:654] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:09.187400 3597 tablet_bootstrap.cc:492] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:09.189386 3596 tablet_bootstrap.cc:492] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:09.189563 3596 ts_tablet_manager.cc:1403] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.003s
I20251212 08:53:09.190270 3596 raft_consensus.cc:359] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:09.190485 3596 raft_consensus.cc:385] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:09.190570 3596 raft_consensus.cc:740] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:09.190795 3596 consensus_queue.cc:260] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:09.191388 3596 ts_tablet_manager.cc:1434] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251212 08:53:09.191411 3597 tablet_bootstrap.cc:654] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:09.194468 3596 tablet_bootstrap.cc:492] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:09.197142 3596 tablet_bootstrap.cc:654] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:09.206393 3597 tablet_bootstrap.cc:492] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:09.206559 3597 ts_tablet_manager.cc:1403] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.008s
I20251212 08:53:09.207439 3597 raft_consensus.cc:359] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:09.207626 3597 raft_consensus.cc:385] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:09.207683 3597 raft_consensus.cc:740] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:09.207727 3599 tablet_bootstrap.cc:492] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:09.207883 3597 consensus_queue.cc:260] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:09.208542 3597 ts_tablet_manager.cc:1434] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:09.208950 3597 tablet_bootstrap.cc:492] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:09.209034 3596 tablet_bootstrap.cc:492] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:09.209245 3596 ts_tablet_manager.cc:1403] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.002s
I20251212 08:53:09.210163 3596 raft_consensus.cc:359] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:09.210429 3596 raft_consensus.cc:385] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:09.210530 3596 raft_consensus.cc:740] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:09.210637 3599 tablet_bootstrap.cc:654] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:09.210780 3597 tablet_bootstrap.cc:654] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:09.210783 3596 consensus_queue.cc:260] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:09.212581 3596 ts_tablet_manager.cc:1434] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251212 08:53:09.219616 3597 tablet_bootstrap.cc:492] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:09.219609 3599 tablet_bootstrap.cc:492] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:09.219826 3597 ts_tablet_manager.cc:1403] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.002s
I20251212 08:53:09.219880 3599 ts_tablet_manager.cc:1403] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.008s
I20251212 08:53:09.221246 3599 raft_consensus.cc:359] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:09.221421 3599 raft_consensus.cc:385] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:09.221357 3597 raft_consensus.cc:359] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:09.221484 3599 raft_consensus.cc:740] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:09.221518 3597 raft_consensus.cc:385] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:09.221573 3597 raft_consensus.cc:740] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:09.221719 3599 consensus_queue.cc:260] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:09.221760 3597 consensus_queue.cc:260] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:09.221880 3572 raft_consensus.cc:493] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:09.221988 3572 raft_consensus.cc:515] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:09.222226 3599 ts_tablet_manager.cc:1434] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:53:09.222527 3599 tablet_bootstrap.cc:492] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:09.222608 3572 leader_election.cc:290] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:09.222867 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7d16ad80c41b41e2accecd66ca20e04a" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:09.223019 2705 raft_consensus.cc:2468] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:09.223093 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7d16ad80c41b41e2accecd66ca20e04a" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:09.223222 2886 raft_consensus.cc:2468] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:09.223510 2763 leader_election.cc:304] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:09.223836 3572 raft_consensus.cc:2804] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:09.223912 3572 raft_consensus.cc:493] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:09.223984 3572 raft_consensus.cc:3060] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:09.224416 3599 tablet_bootstrap.cc:654] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:09.225160 3597 ts_tablet_manager.cc:1434] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20251212 08:53:09.226586 3572 raft_consensus.cc:515] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:09.227542 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7d16ad80c41b41e2accecd66ca20e04a" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:09.227736 2705 raft_consensus.cc:3060] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:09.227710 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "7d16ad80c41b41e2accecd66ca20e04a" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:09.227834 2886 raft_consensus.cc:3060] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:09.230588 2886 raft_consensus.cc:2468] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:09.231097 2765 leader_election.cc:304] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:09.231387 3543 raft_consensus.cc:2804] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:09.232983 3543 raft_consensus.cc:697] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:09.233376 3543 consensus_queue.cc:237] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:09.236363 3572 leader_election.cc:290] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:09.230588 2705 raft_consensus.cc:2468] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:09.247216 3599 tablet_bootstrap.cc:492] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:09.247356 3599 ts_tablet_manager.cc:1403] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.025s user 0.013s sys 0.010s
I20251212 08:53:09.248066 3599 raft_consensus.cc:359] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:09.248206 3599 raft_consensus.cc:385] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:09.248239 3599 raft_consensus.cc:740] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:09.248437 3599 consensus_queue.cc:260] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:09.248418 2585 catalog_manager.cc:5654] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:09.248958 3599 ts_tablet_manager.cc:1434] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:09.411729 3590 raft_consensus.cc:493] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:09.411903 3590 raft_consensus.cc:515] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:09.412549 3590 leader_election.cc:290] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:09.412973 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe34e1fdbf24c1c9c1edc1e14bfa88d" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:09.413043 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe34e1fdbf24c1c9c1edc1e14bfa88d" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:09.413149 2705 raft_consensus.cc:2468] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:09.413244 2807 raft_consensus.cc:2468] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:09.413534 2843 leader_election.cc:304] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:09.413796 3590 raft_consensus.cc:2804] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:09.413877 3590 raft_consensus.cc:493] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:09.413961 3590 raft_consensus.cc:3060] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:09.415817 3590 raft_consensus.cc:515] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:09.416353 3590 leader_election.cc:290] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:09.416774 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe34e1fdbf24c1c9c1edc1e14bfa88d" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:09.416792 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bbe34e1fdbf24c1c9c1edc1e14bfa88d" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:09.416940 2807 raft_consensus.cc:3060] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:09.416987 2702 raft_consensus.cc:3060] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:09.419826 2807 raft_consensus.cc:2468] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:09.419829 2702 raft_consensus.cc:2468] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:09.420454 2845 leader_election.cc:304] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:09.421748 3590 raft_consensus.cc:2804] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:09.421873 3590 raft_consensus.cc:697] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:09.422111 3590 consensus_queue.cc:237] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:09.425663 2589 catalog_manager.cc:5654] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:09.447610 2702 raft_consensus.cc:1275] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:09.447906 2705 raft_consensus.cc:1275] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:09.448091 2807 raft_consensus.cc:1275] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:09.448469 2886 raft_consensus.cc:1275] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:09.448614 3605 consensus_queue.cc:1048] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:09.448400 3590 consensus_queue.cc:1048] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:09.449249 3583 consensus_queue.cc:1048] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:09.449249 3605 consensus_queue.cc:1048] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:09.488473 3610 mvcc.cc:204] Tried to move back new op lower bound from 7231609198373052416 to 7231609197503135744. Current Snapshot: MvccSnapshot[applied={T|T < 7231609198373052416}]
W20251212 08:53:09.831578 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 275ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:10.098589 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 539ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:10.357566 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 806ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:10.493355 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00112e+06us
I20251212 08:53:10.495149 2586 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.21"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:10.495931 2586 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.21 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:10.505573 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 92719eed5a0448f2a99ac8897be6c266 (DEFAULT_TABLE table=table.21 [id=ff63f8e47a724397aa7d44e9df7747b8]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:10.505892 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 92719eed5a0448f2a99ac8897be6c266 (DEFAULT_TABLE table=table.21 [id=ff63f8e47a724397aa7d44e9df7747b8]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:10.506018 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92719eed5a0448f2a99ac8897be6c266. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:10.506220 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92719eed5a0448f2a99ac8897be6c266. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:10.506794 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 9f3f58afa0ad41c6886c0306b7801ea6 (DEFAULT_TABLE table=table.21 [id=ff63f8e47a724397aa7d44e9df7747b8]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:10.507119 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f3f58afa0ad41c6886c0306b7801ea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:10.512040 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 92719eed5a0448f2a99ac8897be6c266 (DEFAULT_TABLE table=table.21 [id=ff63f8e47a724397aa7d44e9df7747b8]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:10.512470 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92719eed5a0448f2a99ac8897be6c266. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:10.514107 3629 tablet_bootstrap.cc:492] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:10.516659 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 9f3f58afa0ad41c6886c0306b7801ea6 (DEFAULT_TABLE table=table.21 [id=ff63f8e47a724397aa7d44e9df7747b8]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:10.516845 3629 tablet_bootstrap.cc:654] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:10.517036 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f3f58afa0ad41c6886c0306b7801ea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:10.517544 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 9f3f58afa0ad41c6886c0306b7801ea6 (DEFAULT_TABLE table=table.21 [id=ff63f8e47a724397aa7d44e9df7747b8]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:10.517858 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9f3f58afa0ad41c6886c0306b7801ea6. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:10.529484 3631 tablet_bootstrap.cc:492] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:10.531844 3630 tablet_bootstrap.cc:492] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:10.538246 3631 tablet_bootstrap.cc:654] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:10.541754 3630 tablet_bootstrap.cc:654] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:10.543978 3629 tablet_bootstrap.cc:492] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:10.544124 3629 ts_tablet_manager.cc:1403] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.030s user 0.005s sys 0.000s
I20251212 08:53:10.544364 3630 tablet_bootstrap.cc:492] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:10.544469 3630 ts_tablet_manager.cc:1403] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.000s
I20251212 08:53:10.544914 3629 raft_consensus.cc:359] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:10.545001 3630 raft_consensus.cc:359] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:10.545083 3629 raft_consensus.cc:385] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:10.545104 3630 raft_consensus.cc:385] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:10.545135 3629 raft_consensus.cc:740] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:10.545150 3630 raft_consensus.cc:740] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:10.545328 3629 consensus_queue.cc:260] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:10.545328 3630 consensus_queue.cc:260] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:10.545825 3629 ts_tablet_manager.cc:1434] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251212 08:53:10.545915 3630 ts_tablet_manager.cc:1434] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:10.546072 3629 tablet_bootstrap.cc:492] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:10.546139 3630 tablet_bootstrap.cc:492] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:10.547899 3629 tablet_bootstrap.cc:654] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:10.548121 3630 tablet_bootstrap.cc:654] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:10.549872 3629 tablet_bootstrap.cc:492] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:10.550053 3629 ts_tablet_manager.cc:1403] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251212 08:53:10.550818 3629 raft_consensus.cc:359] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:10.551086 3629 raft_consensus.cc:385] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:10.551188 3629 raft_consensus.cc:740] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:10.551471 3629 consensus_queue.cc:260] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:10.552026 3629 ts_tablet_manager.cc:1434] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:10.558804 3631 tablet_bootstrap.cc:492] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:10.558946 3631 ts_tablet_manager.cc:1403] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.030s user 0.006s sys 0.000s
I20251212 08:53:10.559545 3631 raft_consensus.cc:359] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:10.559685 3631 raft_consensus.cc:385] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:10.559746 3631 raft_consensus.cc:740] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:10.559921 3631 consensus_queue.cc:260] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:10.560382 3631 ts_tablet_manager.cc:1434] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:10.560859 3631 tablet_bootstrap.cc:492] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:10.562635 3631 tablet_bootstrap.cc:654] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:10.573124 3630 tablet_bootstrap.cc:492] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:10.573346 3630 ts_tablet_manager.cc:1403] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.009s
I20251212 08:53:10.574651 3631 tablet_bootstrap.cc:492] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:10.574827 3631 ts_tablet_manager.cc:1403] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.000s
I20251212 08:53:10.574978 3630 raft_consensus.cc:359] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:10.575177 3630 raft_consensus.cc:385] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:10.575239 3630 raft_consensus.cc:740] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:10.575433 3630 consensus_queue.cc:260] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:10.575596 3631 raft_consensus.cc:359] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:10.575857 3631 raft_consensus.cc:385] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:10.575980 3630 ts_tablet_manager.cc:1434] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:53:10.575920 3631 raft_consensus.cc:740] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:10.576634 3631 consensus_queue.cc:260] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:10.577196 3631 ts_tablet_manager.cc:1434] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:10.704931 3632 raft_consensus.cc:493] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:10.705114 3632 raft_consensus.cc:515] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:10.705817 3632 leader_election.cc:290] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:10.706182 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f3f58afa0ad41c6886c0306b7801ea6" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:10.706212 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f3f58afa0ad41c6886c0306b7801ea6" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:10.706398 2886 raft_consensus.cc:2468] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:10.706398 2807 raft_consensus.cc:2468] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:10.706871 2667 leader_election.cc:304] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:10.707374 3632 raft_consensus.cc:2804] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:10.707463 3632 raft_consensus.cc:493] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:10.707573 3632 raft_consensus.cc:3060] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:10.710337 3632 raft_consensus.cc:515] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:10.710937 3632 leader_election.cc:290] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:10.711310 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f3f58afa0ad41c6886c0306b7801ea6" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:10.711395 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9f3f58afa0ad41c6886c0306b7801ea6" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:10.711474 2886 raft_consensus.cc:3060] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:10.711547 2807 raft_consensus.cc:3060] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:10.713991 2807 raft_consensus.cc:2468] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:10.714099 2886 raft_consensus.cc:2468] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:10.714532 2667 leader_election.cc:304] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:10.714854 3632 raft_consensus.cc:2804] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:10.715119 3632 raft_consensus.cc:697] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:10.715373 3632 consensus_queue.cc:237] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:10.719159 2585 catalog_manager.cc:5654] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:10.720854 3543 raft_consensus.cc:493] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:10.721033 3543 raft_consensus.cc:515] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:10.721640 3543 leader_election.cc:290] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:10.721927 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92719eed5a0448f2a99ac8897be6c266" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:10.722040 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92719eed5a0448f2a99ac8897be6c266" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:10.722132 2886 raft_consensus.cc:2468] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:10.722222 2705 raft_consensus.cc:2468] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:10.722536 2765 leader_election.cc:304] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:10.722771 3543 raft_consensus.cc:2804] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:10.722878 3543 raft_consensus.cc:493] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:10.722983 3543 raft_consensus.cc:3060] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:10.725749 3543 raft_consensus.cc:515] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:10.726321 3543 leader_election.cc:290] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:10.726696 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92719eed5a0448f2a99ac8897be6c266" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:10.726840 2886 raft_consensus.cc:3060] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:10.727080 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "92719eed5a0448f2a99ac8897be6c266" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:10.727268 2705 raft_consensus.cc:3060] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:10.729012 2886 raft_consensus.cc:2468] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:10.729292 2705 raft_consensus.cc:2468] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:10.729393 2765 leader_election.cc:304] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:10.729748 3543 raft_consensus.cc:2804] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:10.729918 3543 raft_consensus.cc:697] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:10.730175 3543 consensus_queue.cc:237] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:10.733249 2585 catalog_manager.cc:5654] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:10.757259 2886 raft_consensus.cc:1275] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:10.757465 2705 raft_consensus.cc:1275] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:10.757527 2885 raft_consensus.cc:1275] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:10.758052 3637 consensus_queue.cc:1048] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:10.758052 3632 consensus_queue.cc:1048] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:10.758359 2807 raft_consensus.cc:1275] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:10.759047 3633 consensus_queue.cc:1048] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:10.761296 3543 consensus_queue.cc:1048] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:11.022315 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 189ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:11.295971 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 453ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:11.556761 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 727ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:11.770381 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00124e+06us
I20251212 08:53:11.771453 2586 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.22"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:11.772452 2586 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.22 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:11.785459 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 50e34e7a14274ee9ad9d640eb15a6909 (DEFAULT_TABLE table=table.22 [id=0a9e4a736bab49149a2fd5b18b4b9951]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:11.785820 2876 tablet_service.cc:1505] Processing CreateTablet for tablet 50e34e7a14274ee9ad9d640eb15a6909 (DEFAULT_TABLE table=table.22 [id=0a9e4a736bab49149a2fd5b18b4b9951]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:11.785959 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e34e7a14274ee9ad9d640eb15a6909. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:11.786185 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e34e7a14274ee9ad9d640eb15a6909. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:11.787022 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 82b6f18e15f941929f188b2b490718ca (DEFAULT_TABLE table=table.22 [id=0a9e4a736bab49149a2fd5b18b4b9951]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:11.787324 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82b6f18e15f941929f188b2b490718ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:11.788705 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 50e34e7a14274ee9ad9d640eb15a6909 (DEFAULT_TABLE table=table.22 [id=0a9e4a736bab49149a2fd5b18b4b9951]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:11.788996 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 50e34e7a14274ee9ad9d640eb15a6909. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:11.792634 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 82b6f18e15f941929f188b2b490718ca (DEFAULT_TABLE table=table.22 [id=0a9e4a736bab49149a2fd5b18b4b9951]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:11.793044 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82b6f18e15f941929f188b2b490718ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:11.795617 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 82b6f18e15f941929f188b2b490718ca (DEFAULT_TABLE table=table.22 [id=0a9e4a736bab49149a2fd5b18b4b9951]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:11.795989 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82b6f18e15f941929f188b2b490718ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:11.809868 3663 tablet_bootstrap.cc:492] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:11.810036 3661 tablet_bootstrap.cc:492] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:11.812860 3661 tablet_bootstrap.cc:654] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:11.813863 3663 tablet_bootstrap.cc:654] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:11.815249 3662 tablet_bootstrap.cc:492] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:11.817389 3662 tablet_bootstrap.cc:654] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:11.819715 3662 tablet_bootstrap.cc:492] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:11.819937 3662 ts_tablet_manager.cc:1403] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20251212 08:53:11.820853 3662 raft_consensus.cc:359] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:11.821014 3662 raft_consensus.cc:385] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:11.821133 3662 raft_consensus.cc:740] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:11.821554 3663 tablet_bootstrap.cc:492] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:11.821477 3662 consensus_queue.cc:260] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:11.821836 3663 ts_tablet_manager.cc:1403] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20251212 08:53:11.822227 3662 ts_tablet_manager.cc:1434] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:11.822562 3662 tablet_bootstrap.cc:492] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:11.822898 3663 raft_consensus.cc:359] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:11.823886 3663 raft_consensus.cc:385] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:11.824045 3663 raft_consensus.cc:740] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:11.824484 3663 consensus_queue.cc:260] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:11.828326 3662 tablet_bootstrap.cc:654] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:11.830145 3663 ts_tablet_manager.cc:1434] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.008s user 0.000s sys 0.003s
I20251212 08:53:11.830595 3663 tablet_bootstrap.cc:492] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:11.830675 3661 tablet_bootstrap.cc:492] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:11.830869 3661 ts_tablet_manager.cc:1403] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.021s user 0.010s sys 0.000s
I20251212 08:53:11.831789 3661 raft_consensus.cc:359] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:11.832067 3661 raft_consensus.cc:385] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:11.832201 3661 raft_consensus.cc:740] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:11.832857 3661 consensus_queue.cc:260] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:11.833485 3663 tablet_bootstrap.cc:654] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:11.833494 3661 ts_tablet_manager.cc:1434] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:11.834141 3661 tablet_bootstrap.cc:492] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:11.836917 3661 tablet_bootstrap.cc:654] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:11.839568 3662 tablet_bootstrap.cc:492] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:11.839757 3662 ts_tablet_manager.cc:1403] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.004s
I20251212 08:53:11.840629 3662 raft_consensus.cc:359] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:11.840899 3662 raft_consensus.cc:385] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:11.841007 3662 raft_consensus.cc:740] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:11.841280 3662 consensus_queue.cc:260] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:11.841825 3662 ts_tablet_manager.cc:1434] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:11.845439 3661 tablet_bootstrap.cc:492] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:11.845602 3661 ts_tablet_manager.cc:1403] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.000s
I20251212 08:53:11.845752 3663 tablet_bootstrap.cc:492] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:11.845973 3663 ts_tablet_manager.cc:1403] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.008s
I20251212 08:53:11.846491 3661 raft_consensus.cc:359] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:11.846680 3661 raft_consensus.cc:385] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:11.846669 3663 raft_consensus.cc:359] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:11.846768 3661 raft_consensus.cc:740] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:11.846837 3663 raft_consensus.cc:385] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:11.846900 3663 raft_consensus.cc:740] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:11.847044 3661 consensus_queue.cc:260] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:11.847118 3663 consensus_queue.cc:260] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:11.847539 3661 ts_tablet_manager.cc:1434] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:11.854336 3663 ts_tablet_manager.cc:1434] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20251212 08:53:11.953060 3656 raft_consensus.cc:493] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:11.953269 3656 raft_consensus.cc:515] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:11.953954 3656 leader_election.cc:290] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:11.954394 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "82b6f18e15f941929f188b2b490718ca" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:11.954365 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "82b6f18e15f941929f188b2b490718ca" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:11.954569 2807 raft_consensus.cc:2468] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:11.954610 2702 raft_consensus.cc:2468] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:11.954937 2845 leader_election.cc:304] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:11.955331 3656 raft_consensus.cc:2804] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:11.955440 3656 raft_consensus.cc:493] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:11.955523 3656 raft_consensus.cc:3060] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:11.957669 3656 raft_consensus.cc:515] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:11.958187 3656 leader_election.cc:290] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:11.958563 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "82b6f18e15f941929f188b2b490718ca" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:11.958607 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "82b6f18e15f941929f188b2b490718ca" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:11.958709 2702 raft_consensus.cc:3060] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:11.958770 2807 raft_consensus.cc:3060] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:11.960942 2807 raft_consensus.cc:2468] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:11.961234 2702 raft_consensus.cc:2468] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:11.961313 2845 leader_election.cc:304] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:11.961483 3633 raft_consensus.cc:493] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:11.961578 3656 raft_consensus.cc:2804] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:11.961807 3656 raft_consensus.cc:697] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:11.962083 3656 consensus_queue.cc:237] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:11.961592 3633 raft_consensus.cc:515] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:11.962813 3633 leader_election.cc:290] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:11.963358 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e34e7a14274ee9ad9d640eb15a6909" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:11.963524 2805 raft_consensus.cc:2468] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:11.963786 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e34e7a14274ee9ad9d640eb15a6909" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:11.963956 2886 raft_consensus.cc:2468] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:11.964095 2667 leader_election.cc:304] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:11.964546 3633 raft_consensus.cc:2804] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:11.964630 3633 raft_consensus.cc:493] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:11.964699 3633 raft_consensus.cc:3060] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:11.966713 2589 catalog_manager.cc:5654] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:11.967072 3633 raft_consensus.cc:515] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:11.967563 3633 leader_election.cc:290] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:11.967911 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e34e7a14274ee9ad9d640eb15a6909" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:11.967931 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "50e34e7a14274ee9ad9d640eb15a6909" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:11.968084 2805 raft_consensus.cc:3060] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:11.968086 2886 raft_consensus.cc:3060] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:11.970057 2805 raft_consensus.cc:2468] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:11.970242 2886 raft_consensus.cc:2468] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:11.970427 2667 leader_election.cc:304] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:11.970642 3633 raft_consensus.cc:2804] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:11.970765 3633 raft_consensus.cc:697] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:11.971035 3633 consensus_queue.cc:237] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:11.974897 2589 catalog_manager.cc:5654] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:12.007452 2702 raft_consensus.cc:1275] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:12.007938 3656 consensus_queue.cc:1048] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:12.008405 2807 raft_consensus.cc:1275] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:12.008481 2886 raft_consensus.cc:1275] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:12.008060 2805 raft_consensus.cc:1275] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:12.008854 3670 consensus_queue.cc:1048] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:12.009100 3633 consensus_queue.cc:1048] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:12.009152 3656 consensus_queue.cc:1048] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:12.327014 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 235ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:12.590425 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 497ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:12.851192 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 760ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:13.031574 2738 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.1:35409, user_credentials={real_user=slave}} blocked reactor thread for 1.00092e+06us
I20251212 08:53:13.032523 2587 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.23"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:13.033427 2587 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.23 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:13.045639 2795 tablet_service.cc:1505] Processing CreateTablet for tablet e389399dd80a4284b150fc473d1928e8 (DEFAULT_TABLE table=table.23 [id=35d7aacb40aa4f70a8c41b7c65776225]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:13.046144 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e389399dd80a4284b150fc473d1928e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:13.046226 2876 tablet_service.cc:1505] Processing CreateTablet for tablet e389399dd80a4284b150fc473d1928e8 (DEFAULT_TABLE table=table.23 [id=35d7aacb40aa4f70a8c41b7c65776225]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:13.046373 2695 tablet_service.cc:1505] Processing CreateTablet for tablet e389399dd80a4284b150fc473d1928e8 (DEFAULT_TABLE table=table.23 [id=35d7aacb40aa4f70a8c41b7c65776225]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:13.046591 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e389399dd80a4284b150fc473d1928e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:13.046695 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e389399dd80a4284b150fc473d1928e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:13.051769 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 8459df9bf95e4b21abb8a96966cb0fb2 (DEFAULT_TABLE table=table.23 [id=35d7aacb40aa4f70a8c41b7c65776225]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:13.052031 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 8459df9bf95e4b21abb8a96966cb0fb2 (DEFAULT_TABLE table=table.23 [id=35d7aacb40aa4f70a8c41b7c65776225]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:13.052167 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8459df9bf95e4b21abb8a96966cb0fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:13.052376 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8459df9bf95e4b21abb8a96966cb0fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:13.053711 3694 tablet_bootstrap.cc:492] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:13.055230 3694 tablet_bootstrap.cc:654] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:13.057309 3694 tablet_bootstrap.cc:492] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:13.057425 3694 ts_tablet_manager.cc:1403] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:13.051769 2793 tablet_service.cc:1505] Processing CreateTablet for tablet 8459df9bf95e4b21abb8a96966cb0fb2 (DEFAULT_TABLE table=table.23 [id=35d7aacb40aa4f70a8c41b7c65776225]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:13.058058 3694 raft_consensus.cc:359] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:13.058205 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8459df9bf95e4b21abb8a96966cb0fb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:13.058248 3694 raft_consensus.cc:385] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:13.058307 3694 raft_consensus.cc:740] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:13.058509 3694 consensus_queue.cc:260] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:13.058954 3694 ts_tablet_manager.cc:1434] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:13.073551 3694 tablet_bootstrap.cc:492] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:13.075680 3694 tablet_bootstrap.cc:654] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:13.076048 3697 tablet_bootstrap.cc:492] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:13.077064 3696 tablet_bootstrap.cc:492] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:13.078622 3697 tablet_bootstrap.cc:654] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:13.078954 3696 tablet_bootstrap.cc:654] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:13.080765 3694 tablet_bootstrap.cc:492] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:13.080932 3694 ts_tablet_manager.cc:1403] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251212 08:53:13.081622 3694 raft_consensus.cc:359] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:13.081760 3694 raft_consensus.cc:385] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:13.081799 3694 raft_consensus.cc:740] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:13.081951 3694 consensus_queue.cc:260] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:13.082382 3694 ts_tablet_manager.cc:1434] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20251212 08:53:13.083741 3696 tablet_bootstrap.cc:492] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:13.083892 3696 ts_tablet_manager.cc:1403] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20251212 08:53:13.084221 3697 tablet_bootstrap.cc:492] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:13.084702 3696 raft_consensus.cc:359] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:13.084914 3696 raft_consensus.cc:385] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:13.085002 3696 raft_consensus.cc:740] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:13.085207 3696 consensus_queue.cc:260] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:13.084401 3697 ts_tablet_manager.cc:1403] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20251212 08:53:13.086861 3697 raft_consensus.cc:359] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:13.087036 3697 raft_consensus.cc:385] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:13.087095 3697 raft_consensus.cc:740] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:13.087314 3697 consensus_queue.cc:260] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:13.087425 3696 ts_tablet_manager.cc:1434] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:13.087803 3696 tablet_bootstrap.cc:492] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:13.087993 3697 ts_tablet_manager.cc:1434] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:13.088219 3697 tablet_bootstrap.cc:492] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:13.089710 3696 tablet_bootstrap.cc:654] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:13.091794 3697 tablet_bootstrap.cc:654] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:13.097373 3696 tablet_bootstrap.cc:492] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:13.097522 3696 ts_tablet_manager.cc:1403] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20251212 08:53:13.097575 3697 tablet_bootstrap.cc:492] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:13.097718 3697 ts_tablet_manager.cc:1403] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.002s
I20251212 08:53:13.098145 3696 raft_consensus.cc:359] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:13.098294 3696 raft_consensus.cc:385] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:13.098258 3697 raft_consensus.cc:359] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:13.098356 3696 raft_consensus.cc:740] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:13.098404 3697 raft_consensus.cc:385] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:13.098469 3697 raft_consensus.cc:740] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:13.098554 3696 consensus_queue.cc:260] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:13.098757 3697 consensus_queue.cc:260] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:13.099268 3696 ts_tablet_manager.cc:1434] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20251212 08:53:13.099285 3697 ts_tablet_manager.cc:1434] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251212 08:53:13.127002 3689 raft_consensus.cc:493] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:13.127158 3689 raft_consensus.cc:515] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:13.127729 3689 leader_election.cc:290] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:13.128114 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e389399dd80a4284b150fc473d1928e8" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:13.128178 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e389399dd80a4284b150fc473d1928e8" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:13.128331 2886 raft_consensus.cc:2468] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:13.128373 2705 raft_consensus.cc:2468] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:13.128746 2765 leader_election.cc:304] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:13.128996 3689 raft_consensus.cc:2804] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:13.129084 3689 raft_consensus.cc:493] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:13.129156 3689 raft_consensus.cc:3060] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:13.133208 3689 raft_consensus.cc:515] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:13.133745 3689 leader_election.cc:290] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:13.134140 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e389399dd80a4284b150fc473d1928e8" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:13.134140 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e389399dd80a4284b150fc473d1928e8" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:13.134380 2705 raft_consensus.cc:3060] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:13.134380 2886 raft_consensus.cc:3060] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:13.137395 2705 raft_consensus.cc:2468] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:13.137395 2886 raft_consensus.cc:2468] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:13.137898 2763 leader_election.cc:304] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:13.138183 3689 raft_consensus.cc:2804] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:13.138305 3689 raft_consensus.cc:697] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:13.138556 3689 consensus_queue.cc:237] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:13.142246 2586 catalog_manager.cc:5654] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
W20251212 08:53:13.173110 2833 tablet.cc:2378] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251212 08:53:13.341200 3684 raft_consensus.cc:493] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:13.341482 3684 raft_consensus.cc:515] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:13.342224 3684 leader_election.cc:290] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:13.342869 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8459df9bf95e4b21abb8a96966cb0fb2" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:13.342881 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8459df9bf95e4b21abb8a96966cb0fb2" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:13.343063 2886 raft_consensus.cc:2468] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:13.343155 2807 raft_consensus.cc:2468] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:13.343576 2667 leader_election.cc:304] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:13.344005 3684 raft_consensus.cc:2804] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:13.344072 3684 raft_consensus.cc:493] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:13.344147 3684 raft_consensus.cc:3060] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:13.346807 3684 raft_consensus.cc:515] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:13.347325 3684 leader_election.cc:290] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:13.347719 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8459df9bf95e4b21abb8a96966cb0fb2" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:13.347755 2886 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "8459df9bf95e4b21abb8a96966cb0fb2" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:13.347911 2807 raft_consensus.cc:3060] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:13.347924 2886 raft_consensus.cc:3060] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:13.350064 2886 raft_consensus.cc:2468] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:13.350209 2807 raft_consensus.cc:2468] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:13.350411 2667 leader_election.cc:304] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:13.350682 3684 raft_consensus.cc:2804] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:13.351128 3684 raft_consensus.cc:697] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:13.351405 3684 consensus_queue.cc:237] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:13.355408 2585 catalog_manager.cc:5654] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:13.431537 2705 raft_consensus.cc:1275] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:13.432140 2886 raft_consensus.cc:1275] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:13.432580 2885 raft_consensus.cc:1275] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:13.432600 3689 consensus_queue.cc:1048] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:13.432860 2807 raft_consensus.cc:1275] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:13.432909 3688 consensus_queue.cc:1048] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:13.433375 3704 consensus_queue.cc:1048] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:13.433578 3684 consensus_queue.cc:1048] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:13.449698 3707 mvcc.cc:204] Tried to move back new op lower bound from 7231609214690881536 to 7231609213497577472. Current Snapshot: MvccSnapshot[applied={T|T < 7231609214690881536}]
W20251212 08:53:13.716475 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 202ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:13.980947 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 465ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:14.242911 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 730ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:14.452507 2738 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.1:35409, user_credentials={real_user=slave}} blocked reactor thread for 1.00117e+06us
I20251212 08:53:14.453696 2587 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.24"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:14.455044 2587 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.24 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:14.464382 2795 tablet_service.cc:1505] Processing CreateTablet for tablet de0691c0e98f4700b784a2981f741b5a (DEFAULT_TABLE table=table.24 [id=2ceb6211fa9142ff95a444c67013a466]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:14.464680 2873 tablet_service.cc:1505] Processing CreateTablet for tablet de0691c0e98f4700b784a2981f741b5a (DEFAULT_TABLE table=table.24 [id=2ceb6211fa9142ff95a444c67013a466]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:14.464881 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0691c0e98f4700b784a2981f741b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:14.465018 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0691c0e98f4700b784a2981f741b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:14.470827 2695 tablet_service.cc:1505] Processing CreateTablet for tablet de0691c0e98f4700b784a2981f741b5a (DEFAULT_TABLE table=table.24 [id=2ceb6211fa9142ff95a444c67013a466]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:14.471266 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet de0691c0e98f4700b784a2981f741b5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:14.471591 2694 tablet_service.cc:1505] Processing CreateTablet for tablet d7e15336531948be810a2944afa986a4 (DEFAULT_TABLE table=table.24 [id=2ceb6211fa9142ff95a444c67013a466]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:14.472018 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7e15336531948be810a2944afa986a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:14.478021 2793 tablet_service.cc:1505] Processing CreateTablet for tablet d7e15336531948be810a2944afa986a4 (DEFAULT_TABLE table=table.24 [id=2ceb6211fa9142ff95a444c67013a466]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:14.478431 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7e15336531948be810a2944afa986a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:14.479585 2876 tablet_service.cc:1505] Processing CreateTablet for tablet d7e15336531948be810a2944afa986a4 (DEFAULT_TABLE table=table.24 [id=2ceb6211fa9142ff95a444c67013a466]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:14.479938 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7e15336531948be810a2944afa986a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:14.487875 3726 tablet_bootstrap.cc:492] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:14.490890 3726 tablet_bootstrap.cc:654] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:14.494709 3724 tablet_bootstrap.cc:492] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:14.499203 3724 tablet_bootstrap.cc:654] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:14.501713 3725 tablet_bootstrap.cc:492] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:14.506724 3725 tablet_bootstrap.cc:654] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:14.502945 3726 tablet_bootstrap.cc:492] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:14.507825 3726 ts_tablet_manager.cc:1403] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.009s
I20251212 08:53:14.508824 3726 raft_consensus.cc:359] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:14.509198 3726 raft_consensus.cc:385] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:14.509310 3726 raft_consensus.cc:740] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:14.509570 3726 consensus_queue.cc:260] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:14.510226 3726 ts_tablet_manager.cc:1434] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:14.509704 3725 tablet_bootstrap.cc:492] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:14.510409 3725 ts_tablet_manager.cc:1403] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20251212 08:53:14.511195 3725 raft_consensus.cc:359] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:14.511587 3726 tablet_bootstrap.cc:492] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:14.511643 3724 tablet_bootstrap.cc:492] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:14.511859 3724 ts_tablet_manager.cc:1403] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.000s
I20251212 08:53:14.512702 3724 raft_consensus.cc:359] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:14.512959 3724 raft_consensus.cc:385] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:14.513060 3724 raft_consensus.cc:740] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:14.513306 3724 consensus_queue.cc:260] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:14.513584 3726 tablet_bootstrap.cc:654] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:14.513943 3724 ts_tablet_manager.cc:1434] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:14.511657 3725 raft_consensus.cc:385] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:14.514667 3725 raft_consensus.cc:740] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:14.514931 3725 consensus_queue.cc:260] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:14.515543 3725 ts_tablet_manager.cc:1434] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s
I20251212 08:53:14.515616 3724 tablet_bootstrap.cc:492] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:14.515991 3725 tablet_bootstrap.cc:492] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:14.517735 3724 tablet_bootstrap.cc:654] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:14.518383 3725 tablet_bootstrap.cc:654] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:14.524169 3725 tablet_bootstrap.cc:492] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:14.524415 3725 ts_tablet_manager.cc:1403] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20251212 08:53:14.525610 3725 raft_consensus.cc:359] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:14.525898 3725 raft_consensus.cc:385] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:14.525981 3725 raft_consensus.cc:740] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:14.526226 3725 consensus_queue.cc:260] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:14.526924 3725 ts_tablet_manager.cc:1434] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251212 08:53:14.528927 3726 tablet_bootstrap.cc:492] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:14.529124 3726 ts_tablet_manager.cc:1403] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.018s user 0.005s sys 0.000s
I20251212 08:53:14.530105 3726 raft_consensus.cc:359] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:14.530341 3726 raft_consensus.cc:385] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:14.530417 3726 raft_consensus.cc:740] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:14.530648 3726 consensus_queue.cc:260] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:14.531411 3726 ts_tablet_manager.cc:1434] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251212 08:53:14.532819 3724 tablet_bootstrap.cc:492] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:14.532974 3724 ts_tablet_manager.cc:1403] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.007s
I20251212 08:53:14.533857 3724 raft_consensus.cc:359] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:14.534046 3724 raft_consensus.cc:385] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:14.534106 3724 raft_consensus.cc:740] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:14.534662 3708 raft_consensus.cc:493] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:14.534786 3708 raft_consensus.cc:515] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:14.535392 3708 leader_election.cc:290] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:14.536019 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d7e15336531948be810a2944afa986a4" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:14.536466 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d7e15336531948be810a2944afa986a4" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:14.536654 2885 raft_consensus.cc:2468] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:14.534363 3724 consensus_queue.cc:260] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:14.537043 2805 raft_consensus.cc:2468] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:14.537159 2667 leader_election.cc:304] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:14.537807 3724 ts_tablet_manager.cc:1434] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251212 08:53:14.538131 3708 raft_consensus.cc:2804] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:14.538228 3708 raft_consensus.cc:493] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:14.538304 3708 raft_consensus.cc:3060] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:14.541034 3708 raft_consensus.cc:515] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:14.541604 3708 leader_election.cc:290] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:14.542124 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d7e15336531948be810a2944afa986a4" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:14.542308 2885 raft_consensus.cc:3060] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:14.543074 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d7e15336531948be810a2944afa986a4" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:14.543244 2805 raft_consensus.cc:3060] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:14.545686 2805 raft_consensus.cc:2468] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:14.546051 2667 leader_election.cc:304] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:14.546305 3708 raft_consensus.cc:2804] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:14.546718 3708 raft_consensus.cc:697] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:14.546988 3708 consensus_queue.cc:237] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:14.557765 2586 catalog_manager.cc:5654] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:14.571329 2885 raft_consensus.cc:2468] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:14.611296 3735 raft_consensus.cc:493] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:14.611464 3735 raft_consensus.cc:515] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:14.612082 3735 leader_election.cc:290] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:14.612531 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de0691c0e98f4700b784a2981f741b5a" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:14.612771 2805 raft_consensus.cc:2468] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:14.612524 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de0691c0e98f4700b784a2981f741b5a" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:14.613121 2885 raft_consensus.cc:2468] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:14.613215 2667 leader_election.cc:304] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:14.613489 3735 raft_consensus.cc:2804] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:14.613590 3735 raft_consensus.cc:493] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:14.613667 3735 raft_consensus.cc:3060] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:14.615710 3735 raft_consensus.cc:515] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:14.616222 3735 leader_election.cc:290] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:14.616765 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de0691c0e98f4700b784a2981f741b5a" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:14.616765 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "de0691c0e98f4700b784a2981f741b5a" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:14.617005 2885 raft_consensus.cc:3060] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:14.617008 2805 raft_consensus.cc:3060] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:14.619863 2885 raft_consensus.cc:2468] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:14.619864 2805 raft_consensus.cc:2468] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:14.620411 2667 leader_election.cc:304] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:14.621027 3735 raft_consensus.cc:2804] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:14.621188 3735 raft_consensus.cc:697] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:14.621454 3735 consensus_queue.cc:237] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:14.626955 2586 catalog_manager.cc:5654] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:14.640897 2805 raft_consensus.cc:1275] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:14.640990 2887 raft_consensus.cc:1275] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:14.640875 2885 raft_consensus.cc:1275] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:14.641474 2807 raft_consensus.cc:1275] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:14.641557 3733 consensus_queue.cc:1048] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:14.641878 3735 consensus_queue.cc:1048] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:14.642263 3708 consensus_queue.cc:1048] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:14.642385 3735 consensus_queue.cc:1048] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:15.012784 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 294ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:15.277906 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 554ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:15.540086 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 819ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:15.660081 2738 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.1:35409, user_credentials={real_user=slave}} blocked reactor thread for 1.00092e+06us
I20251212 08:53:15.661679 2585 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.25"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:15.662659 2585 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.25 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:15.675817 2876 tablet_service.cc:1505] Processing CreateTablet for tablet e66ba33ab78f4604b175eef3351f38c1 (DEFAULT_TABLE table=table.25 [id=37811fd24a724edfa972b026a3a77795]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:15.676261 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66ba33ab78f4604b175eef3351f38c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:15.677044 2694 tablet_service.cc:1505] Processing CreateTablet for tablet e66ba33ab78f4604b175eef3351f38c1 (DEFAULT_TABLE table=table.25 [id=37811fd24a724edfa972b026a3a77795]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:15.677608 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 4802e69098d74c6eba89068ae8a73106 (DEFAULT_TABLE table=table.25 [id=37811fd24a724edfa972b026a3a77795]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:15.677908 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4802e69098d74c6eba89068ae8a73106. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:15.678557 2793 tablet_service.cc:1505] Processing CreateTablet for tablet e66ba33ab78f4604b175eef3351f38c1 (DEFAULT_TABLE table=table.25 [id=37811fd24a724edfa972b026a3a77795]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:15.678862 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66ba33ab78f4604b175eef3351f38c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:15.684209 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 4802e69098d74c6eba89068ae8a73106 (DEFAULT_TABLE table=table.25 [id=37811fd24a724edfa972b026a3a77795]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:15.684911 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 4802e69098d74c6eba89068ae8a73106 (DEFAULT_TABLE table=table.25 [id=37811fd24a724edfa972b026a3a77795]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:15.685328 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4802e69098d74c6eba89068ae8a73106. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:15.685861 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66ba33ab78f4604b175eef3351f38c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:15.693638 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4802e69098d74c6eba89068ae8a73106. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:15.719034 3759 tablet_bootstrap.cc:492] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:15.729362 3759 tablet_bootstrap.cc:654] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:15.736016 3761 tablet_bootstrap.cc:492] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:15.737576 3761 tablet_bootstrap.cc:654] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:15.738652 3758 tablet_bootstrap.cc:492] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:15.740089 3758 tablet_bootstrap.cc:654] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:15.741880 3761 tablet_bootstrap.cc:492] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:15.743759 3761 ts_tablet_manager.cc:1403] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.001s
I20251212 08:53:15.745811 3761 raft_consensus.cc:359] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:15.745985 3761 raft_consensus.cc:385] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:15.747331 3761 raft_consensus.cc:740] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:15.748030 3761 consensus_queue.cc:260] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:15.751763 3759 tablet_bootstrap.cc:492] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:15.752194 3759 ts_tablet_manager.cc:1403] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.033s user 0.003s sys 0.008s
I20251212 08:53:15.753258 3759 raft_consensus.cc:359] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:15.753433 3759 raft_consensus.cc:385] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:15.753504 3759 raft_consensus.cc:740] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:15.753692 3759 consensus_queue.cc:260] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:15.754096 3759 ts_tablet_manager.cc:1434] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251212 08:53:15.754851 3761 ts_tablet_manager.cc:1434] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.010s user 0.000s sys 0.001s
I20251212 08:53:15.759282 3761 tablet_bootstrap.cc:492] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:15.760562 3758 tablet_bootstrap.cc:492] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:15.760751 3758 ts_tablet_manager.cc:1403] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.022s user 0.008s sys 0.014s
I20251212 08:53:15.761487 3758 raft_consensus.cc:359] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:15.761813 3758 raft_consensus.cc:385] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:15.761927 3758 raft_consensus.cc:740] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:15.762180 3758 consensus_queue.cc:260] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:15.762319 3761 tablet_bootstrap.cc:654] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:15.762782 3758 ts_tablet_manager.cc:1434] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251212 08:53:15.763099 3758 tablet_bootstrap.cc:492] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:15.759272 3759 tablet_bootstrap.cc:492] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:15.765098 3759 tablet_bootstrap.cc:654] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:15.766610 3758 tablet_bootstrap.cc:654] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:15.766906 3761 tablet_bootstrap.cc:492] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:15.767066 3761 ts_tablet_manager.cc:1403] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20251212 08:53:15.767889 3761 raft_consensus.cc:359] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:15.768136 3761 raft_consensus.cc:385] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:15.768231 3761 raft_consensus.cc:740] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:15.768500 3758 tablet_bootstrap.cc:492] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:15.768455 3761 consensus_queue.cc:260] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:15.768620 3758 ts_tablet_manager.cc:1403] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20251212 08:53:15.768970 3761 ts_tablet_manager.cc:1434] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:15.769217 3758 raft_consensus.cc:359] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:15.769340 3758 raft_consensus.cc:385] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:15.769429 3758 raft_consensus.cc:740] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:15.769658 3758 consensus_queue.cc:260] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:15.769927 3759 tablet_bootstrap.cc:492] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:15.770051 3759 ts_tablet_manager.cc:1403] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.003s
I20251212 08:53:15.770884 3759 raft_consensus.cc:359] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:15.771042 3759 raft_consensus.cc:385] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:15.771106 3759 raft_consensus.cc:740] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:15.771416 3758 ts_tablet_manager.cc:1434] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:15.771337 3759 consensus_queue.cc:260] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:15.776361 3759 ts_tablet_manager.cc:1434] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20251212 08:53:15.876140 3708 raft_consensus.cc:493] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:15.876351 3708 raft_consensus.cc:515] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:15.876936 3708 leader_election.cc:290] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:15.877447 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4802e69098d74c6eba89068ae8a73106" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:15.877480 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4802e69098d74c6eba89068ae8a73106" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:15.877588 2807 raft_consensus.cc:2468] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:15.877669 2887 raft_consensus.cc:2468] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:15.878144 2667 leader_election.cc:304] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:15.878355 3708 raft_consensus.cc:2804] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:15.878460 3708 raft_consensus.cc:493] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:15.878548 3708 raft_consensus.cc:3060] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:15.881388 3708 raft_consensus.cc:515] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:15.882018 3708 leader_election.cc:290] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:15.882413 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4802e69098d74c6eba89068ae8a73106" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:15.882490 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4802e69098d74c6eba89068ae8a73106" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:15.882601 2887 raft_consensus.cc:3060] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:15.882674 2807 raft_consensus.cc:3060] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:15.884864 2887 raft_consensus.cc:2468] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:15.885164 2807 raft_consensus.cc:2468] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:15.885268 2667 leader_election.cc:304] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:15.885682 3708 raft_consensus.cc:2804] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:15.885854 3708 raft_consensus.cc:697] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:15.886111 3708 consensus_queue.cc:237] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:15.889511 2589 catalog_manager.cc:5654] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:15.908329 3760 raft_consensus.cc:493] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:15.908531 3760 raft_consensus.cc:515] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:15.909265 3760 leader_election.cc:290] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:15.909565 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e66ba33ab78f4604b175eef3351f38c1" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:15.909752 2887 raft_consensus.cc:2468] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:15.909673 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e66ba33ab78f4604b175eef3351f38c1" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:15.909940 2705 raft_consensus.cc:2468] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:15.910148 2765 leader_election.cc:304] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:15.910413 3760 raft_consensus.cc:2804] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:15.910496 3760 raft_consensus.cc:493] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:15.910586 3760 raft_consensus.cc:3060] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:15.912593 3760 raft_consensus.cc:515] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:15.913100 3760 leader_election.cc:290] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:15.913424 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e66ba33ab78f4604b175eef3351f38c1" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:15.913488 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e66ba33ab78f4604b175eef3351f38c1" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:15.913599 2887 raft_consensus.cc:3060] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:15.913690 2705 raft_consensus.cc:3060] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:15.915613 2705 raft_consensus.cc:2468] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:15.915609 2887 raft_consensus.cc:2468] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:15.915968 2763 leader_election.cc:304] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:15.916241 3760 raft_consensus.cc:2804] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:15.916425 3760 raft_consensus.cc:697] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:15.916677 3760 consensus_queue.cc:237] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:15.920442 2585 catalog_manager.cc:5654] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:15.938383 2887 raft_consensus.cc:1275] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:15.938612 2807 raft_consensus.cc:1275] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:15.938735 2885 raft_consensus.cc:1275] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:15.938831 2705 raft_consensus.cc:1275] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:15.939055 3752 consensus_queue.cc:1048] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:15.939321 3760 consensus_queue.cc:1048] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:15.939344 3708 consensus_queue.cc:1048] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:15.939620 3689 consensus_queue.cc:1048] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:16.201363 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 180ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:16.465571 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 440ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:16.729257 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 704ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:16.962947 2738 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.1:35409, user_credentials={real_user=slave}} blocked reactor thread for 1.00113e+06us
I20251212 08:53:16.964248 2588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.26"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:16.965991 2588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.26 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:16.980430 2793 tablet_service.cc:1505] Processing CreateTablet for tablet c3c476911f1449d299b316d4d5730c71 (DEFAULT_TABLE table=table.26 [id=9d06e9cd6a5f41a38504eea36ecee19c]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:16.980859 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c476911f1449d299b316d4d5730c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:16.981248 2694 tablet_service.cc:1505] Processing CreateTablet for tablet c3c476911f1449d299b316d4d5730c71 (DEFAULT_TABLE table=table.26 [id=9d06e9cd6a5f41a38504eea36ecee19c]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:16.981601 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c476911f1449d299b316d4d5730c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:16.982544 2695 tablet_service.cc:1505] Processing CreateTablet for tablet bc958968094c4914ae8e620f1cfb94fa (DEFAULT_TABLE table=table.26 [id=9d06e9cd6a5f41a38504eea36ecee19c]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:16.982936 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc958968094c4914ae8e620f1cfb94fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:16.986512 2876 tablet_service.cc:1505] Processing CreateTablet for tablet c3c476911f1449d299b316d4d5730c71 (DEFAULT_TABLE table=table.26 [id=9d06e9cd6a5f41a38504eea36ecee19c]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:16.986765 2873 tablet_service.cc:1505] Processing CreateTablet for tablet bc958968094c4914ae8e620f1cfb94fa (DEFAULT_TABLE table=table.26 [id=9d06e9cd6a5f41a38504eea36ecee19c]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:16.986891 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3c476911f1449d299b316d4d5730c71. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:16.987061 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc958968094c4914ae8e620f1cfb94fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:16.987414 3789 tablet_bootstrap.cc:492] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:16.986512 2795 tablet_service.cc:1505] Processing CreateTablet for tablet bc958968094c4914ae8e620f1cfb94fa (DEFAULT_TABLE table=table.26 [id=9d06e9cd6a5f41a38504eea36ecee19c]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:16.990367 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc958968094c4914ae8e620f1cfb94fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:17.001559 3791 tablet_bootstrap.cc:492] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:17.003710 3791 tablet_bootstrap.cc:654] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:17.004099 3790 tablet_bootstrap.cc:492] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:17.007169 3790 tablet_bootstrap.cc:654] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:17.007817 3789 tablet_bootstrap.cc:654] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:17.012617 3791 tablet_bootstrap.cc:492] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:17.012631 3790 tablet_bootstrap.cc:492] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:17.012786 3791 ts_tablet_manager.cc:1403] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.005s
I20251212 08:53:17.012786 3790 ts_tablet_manager.cc:1403] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20251212 08:53:17.013722 3790 raft_consensus.cc:359] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:17.013960 3791 raft_consensus.cc:359] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:17.014019 3790 raft_consensus.cc:385] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:17.014091 3791 raft_consensus.cc:385] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:17.014117 3790 raft_consensus.cc:740] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:17.014142 3791 raft_consensus.cc:740] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:17.014312 3790 consensus_queue.cc:260] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:17.014312 3791 consensus_queue.cc:260] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:17.014814 3790 ts_tablet_manager.cc:1434] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:17.015075 3791 ts_tablet_manager.cc:1434] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:17.015360 3791 tablet_bootstrap.cc:492] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:17.015600 3789 tablet_bootstrap.cc:492] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:17.015712 3789 ts_tablet_manager.cc:1403] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.028s user 0.002s sys 0.002s
I20251212 08:53:17.016230 3790 tablet_bootstrap.cc:492] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:17.016523 3789 raft_consensus.cc:359] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:17.016686 3789 raft_consensus.cc:385] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:17.016737 3789 raft_consensus.cc:740] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:17.016920 3789 consensus_queue.cc:260] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:17.017468 3789 ts_tablet_manager.cc:1434] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:17.017694 3789 tablet_bootstrap.cc:492] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:17.017949 3790 tablet_bootstrap.cc:654] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:17.018788 3791 tablet_bootstrap.cc:654] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:17.019295 3789 tablet_bootstrap.cc:654] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:17.039122 3790 tablet_bootstrap.cc:492] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:17.039297 3790 ts_tablet_manager.cc:1403] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.000s
I20251212 08:53:17.039489 3791 tablet_bootstrap.cc:492] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:17.039645 3791 ts_tablet_manager.cc:1403] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.024s user 0.003s sys 0.010s
I20251212 08:53:17.039950 3789 tablet_bootstrap.cc:492] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:17.040064 3789 ts_tablet_manager.cc:1403] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.022s user 0.009s sys 0.011s
I20251212 08:53:17.040136 3790 raft_consensus.cc:359] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:17.040356 3790 raft_consensus.cc:385] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:17.040421 3790 raft_consensus.cc:740] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:17.040619 3790 consensus_queue.cc:260] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:17.040771 3789 raft_consensus.cc:359] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:17.040936 3789 raft_consensus.cc:385] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:17.040992 3789 raft_consensus.cc:740] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:17.041126 3790 ts_tablet_manager.cc:1434] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:17.041199 3789 consensus_queue.cc:260] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:17.041340 3791 raft_consensus.cc:359] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:17.041519 3791 raft_consensus.cc:385] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:17.041607 3791 raft_consensus.cc:740] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:17.041646 3789 ts_tablet_manager.cc:1434] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:17.041855 3791 consensus_queue.cc:260] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:17.042338 3791 ts_tablet_manager.cc:1434] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:53:17.154704 3708 raft_consensus.cc:493] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:17.154913 3708 raft_consensus.cc:515] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:17.155447 3708 leader_election.cc:290] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:17.155884 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3c476911f1449d299b316d4d5730c71" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:17.155979 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3c476911f1449d299b316d4d5730c71" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:17.156065 2807 raft_consensus.cc:2468] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:17.156143 2887 raft_consensus.cc:2468] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:17.156509 2667 leader_election.cc:304] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:17.156787 3708 raft_consensus.cc:2804] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:17.156868 3708 raft_consensus.cc:493] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:17.156944 3708 raft_consensus.cc:3060] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:17.158818 3708 raft_consensus.cc:515] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:17.159272 3708 leader_election.cc:290] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:17.159718 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3c476911f1449d299b316d4d5730c71" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:17.159718 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c3c476911f1449d299b316d4d5730c71" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:17.159926 2807 raft_consensus.cc:3060] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:17.159943 2885 raft_consensus.cc:3060] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:17.162755 2807 raft_consensus.cc:2468] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:17.162755 2885 raft_consensus.cc:2468] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:17.163168 2667 leader_election.cc:304] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:17.163408 3708 raft_consensus.cc:2804] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:17.163578 3708 raft_consensus.cc:697] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:17.163841 3708 consensus_queue.cc:237] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:17.169188 2585 catalog_manager.cc:5654] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:17.306665 3736 raft_consensus.cc:493] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:17.306838 3736 raft_consensus.cc:515] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:17.307552 3736 leader_election.cc:290] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:17.307907 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc958968094c4914ae8e620f1cfb94fa" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:17.308101 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc958968094c4914ae8e620f1cfb94fa" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:17.308116 2705 raft_consensus.cc:2468] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:17.308306 2885 raft_consensus.cc:2468] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:17.308666 2763 leader_election.cc:304] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:17.308936 3736 raft_consensus.cc:2804] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:17.309026 3736 raft_consensus.cc:493] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:17.309108 3736 raft_consensus.cc:3060] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:17.311672 3736 raft_consensus.cc:515] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:17.312237 3736 leader_election.cc:290] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:17.312619 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc958968094c4914ae8e620f1cfb94fa" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:17.312774 2705 raft_consensus.cc:3060] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:17.312778 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "bc958968094c4914ae8e620f1cfb94fa" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:17.312992 2885 raft_consensus.cc:3060] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:17.316442 2705 raft_consensus.cc:2468] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:17.316913 2763 leader_election.cc:304] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:17.317143 3760 raft_consensus.cc:2804] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:17.317808 3760 raft_consensus.cc:697] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:17.318099 3760 consensus_queue.cc:237] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:17.324038 2589 catalog_manager.cc:5654] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:17.347209 2885 raft_consensus.cc:2468] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:17.363262 2885 raft_consensus.cc:1275] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:17.363454 2807 raft_consensus.cc:1275] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:17.363857 3708 consensus_queue.cc:1048] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:17.364106 3752 consensus_queue.cc:1048] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:17.367082 2887 raft_consensus.cc:1275] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:17.367542 2702 raft_consensus.cc:1275] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:17.367815 3760 consensus_queue.cc:1048] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:17.368265 3760 consensus_queue.cc:1048] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:17.389219 3806 mvcc.cc:204] Tried to move back new op lower bound from 7231609230809989120 to 7231609230617165824. Current Snapshot: MvccSnapshot[applied={T|T < 7231609230809989120}]
W20251212 08:53:17.679003 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 224ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:17.949011 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 489ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:18.210096 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 759ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:18.390846 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.0009e+06us
I20251212 08:53:18.391798 2588 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.27"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:18.392760 2588 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.27 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:18.404091 2795 tablet_service.cc:1505] Processing CreateTablet for tablet c0aa78ae4efc4856a0262730d672874f (DEFAULT_TABLE table=table.27 [id=089c3d77be1140e191129897e41cfe07]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:18.404583 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0aa78ae4efc4856a0262730d672874f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:18.404759 2695 tablet_service.cc:1505] Processing CreateTablet for tablet c0aa78ae4efc4856a0262730d672874f (DEFAULT_TABLE table=table.27 [id=089c3d77be1140e191129897e41cfe07]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:18.405064 2694 tablet_service.cc:1505] Processing CreateTablet for tablet c57c808afa054124984ee62dcaa4c22e (DEFAULT_TABLE table=table.27 [id=089c3d77be1140e191129897e41cfe07]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:18.405216 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0aa78ae4efc4856a0262730d672874f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:18.405390 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c57c808afa054124984ee62dcaa4c22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:18.409050 2793 tablet_service.cc:1505] Processing CreateTablet for tablet c57c808afa054124984ee62dcaa4c22e (DEFAULT_TABLE table=table.27 [id=089c3d77be1140e191129897e41cfe07]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:18.409590 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c57c808afa054124984ee62dcaa4c22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:18.409919 2873 tablet_service.cc:1505] Processing CreateTablet for tablet c57c808afa054124984ee62dcaa4c22e (DEFAULT_TABLE table=table.27 [id=089c3d77be1140e191129897e41cfe07]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:18.410264 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c57c808afa054124984ee62dcaa4c22e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:18.411020 2876 tablet_service.cc:1505] Processing CreateTablet for tablet c0aa78ae4efc4856a0262730d672874f (DEFAULT_TABLE table=table.27 [id=089c3d77be1140e191129897e41cfe07]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:18.411357 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0aa78ae4efc4856a0262730d672874f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:18.432646 3821 tablet_bootstrap.cc:492] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:18.434654 3819 tablet_bootstrap.cc:492] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:18.436908 3820 tablet_bootstrap.cc:492] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:18.437382 3819 tablet_bootstrap.cc:654] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:18.435485 3821 tablet_bootstrap.cc:654] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:18.439803 3820 tablet_bootstrap.cc:654] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:18.440429 3821 tablet_bootstrap.cc:492] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:18.440567 3821 ts_tablet_manager.cc:1403] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.003s
I20251212 08:53:18.440856 3819 tablet_bootstrap.cc:492] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:18.440999 3819 ts_tablet_manager.cc:1403] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20251212 08:53:18.441568 3819 raft_consensus.cc:359] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:18.441767 3819 raft_consensus.cc:385] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:18.441591 3821 raft_consensus.cc:359] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:18.441958 3821 raft_consensus.cc:385] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:18.442060 3821 raft_consensus.cc:740] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:18.442277 3821 consensus_queue.cc:260] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:18.442936 3821 ts_tablet_manager.cc:1434] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:18.441845 3819 raft_consensus.cc:740] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:18.443267 3820 tablet_bootstrap.cc:492] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:18.443439 3820 ts_tablet_manager.cc:1403] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20251212 08:53:18.443625 3821 tablet_bootstrap.cc:492] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:18.444195 3820 raft_consensus.cc:359] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:18.444509 3820 raft_consensus.cc:385] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:18.446061 3820 raft_consensus.cc:740] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:18.446321 3820 consensus_queue.cc:260] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:18.445199 3821 tablet_bootstrap.cc:654] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:18.446861 3820 ts_tablet_manager.cc:1434] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:53:18.447095 3820 tablet_bootstrap.cc:492] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:18.443284 3819 consensus_queue.cc:260] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:18.447945 3819 ts_tablet_manager.cc:1434] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.007s user 0.000s sys 0.002s
I20251212 08:53:18.448268 3819 tablet_bootstrap.cc:492] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:18.448764 3820 tablet_bootstrap.cc:654] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:18.450018 3819 tablet_bootstrap.cc:654] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:18.458709 3819 tablet_bootstrap.cc:492] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:18.458904 3819 ts_tablet_manager.cc:1403] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20251212 08:53:18.459527 3819 raft_consensus.cc:359] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:18.459677 3819 raft_consensus.cc:385] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:18.459736 3819 raft_consensus.cc:740] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:18.459957 3819 consensus_queue.cc:260] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:18.460489 3819 ts_tablet_manager.cc:1434] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:18.463210 3820 tablet_bootstrap.cc:492] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:18.463222 3821 tablet_bootstrap.cc:492] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:18.463389 3820 ts_tablet_manager.cc:1403] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.007s
I20251212 08:53:18.463433 3821 ts_tablet_manager.cc:1403] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.020s user 0.001s sys 0.006s
I20251212 08:53:18.464059 3821 raft_consensus.cc:359] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:18.464197 3821 raft_consensus.cc:385] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:18.464141 3820 raft_consensus.cc:359] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:18.464308 3821 raft_consensus.cc:740] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:18.464344 3820 raft_consensus.cc:385] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:18.464406 3820 raft_consensus.cc:740] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:18.464525 3821 consensus_queue.cc:260] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:18.464588 3820 consensus_queue.cc:260] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:18.465037 3821 ts_tablet_manager.cc:1434] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:18.465049 3820 ts_tablet_manager.cc:1434] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:53:18.498569 3784 raft_consensus.cc:493] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:18.498764 3784 raft_consensus.cc:515] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:18.499436 3784 leader_election.cc:290] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:18.499917 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c57c808afa054124984ee62dcaa4c22e" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:18.500167 2805 raft_consensus.cc:2468] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:18.500519 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c57c808afa054124984ee62dcaa4c22e" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:18.500658 2667 leader_election.cc:304] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:18.500684 2885 raft_consensus.cc:2468] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:18.500979 3784 raft_consensus.cc:2804] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:18.501075 3784 raft_consensus.cc:493] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:18.501153 3784 raft_consensus.cc:3060] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:18.503427 3784 raft_consensus.cc:515] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:18.503993 3784 leader_election.cc:290] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:18.504259 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c57c808afa054124984ee62dcaa4c22e" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:18.504453 2805 raft_consensus.cc:3060] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:18.507109 2805 raft_consensus.cc:2468] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:18.507458 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c57c808afa054124984ee62dcaa4c22e" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:18.507468 2667 leader_election.cc:304] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:18.507627 2885 raft_consensus.cc:3060] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:18.507732 3784 raft_consensus.cc:2804] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:18.507866 3784 raft_consensus.cc:697] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:18.508118 3784 consensus_queue.cc:237] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:18.510398 2885 raft_consensus.cc:2468] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:18.515969 2589 catalog_manager.cc:5654] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:18.752153 3828 raft_consensus.cc:493] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:18.752363 3828 raft_consensus.cc:515] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:18.752950 3828 leader_election.cc:290] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:18.753297 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0aa78ae4efc4856a0262730d672874f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:18.753345 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0aa78ae4efc4856a0262730d672874f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:18.753466 2805 raft_consensus.cc:2468] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:18.753624 2704 raft_consensus.cc:2468] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:18.753933 2845 leader_election.cc:304] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:18.754307 3828 raft_consensus.cc:2804] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:18.754385 3828 raft_consensus.cc:493] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:18.754446 3828 raft_consensus.cc:3060] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:18.756654 3828 raft_consensus.cc:515] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:18.757182 3828 leader_election.cc:290] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:18.757607 2704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0aa78ae4efc4856a0262730d672874f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:18.757745 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c0aa78ae4efc4856a0262730d672874f" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:18.757921 2805 raft_consensus.cc:3060] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:18.760066 2805 raft_consensus.cc:2468] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:18.757764 2704 raft_consensus.cc:3060] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:18.760449 2845 leader_election.cc:304] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:18.760715 3783 raft_consensus.cc:2804] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:18.763428 2704 raft_consensus.cc:2468] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:18.764504 3783 raft_consensus.cc:697] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:18.764770 3783 consensus_queue.cc:237] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:18.769315 2588 catalog_manager.cc:5654] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:18.792199 2887 raft_consensus.cc:1275] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:18.792243 2807 raft_consensus.cc:1275] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:18.792820 3752 consensus_queue.cc:1048] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:18.793191 3752 consensus_queue.cc:1048] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:18.794777 2805 raft_consensus.cc:1275] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:18.795190 2704 raft_consensus.cc:1275] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:18.795353 3830 consensus_queue.cc:1048] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:18.795765 3783 consensus_queue.cc:1048] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:19.074299 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 202ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:19.337162 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 464ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:19.600826 836 kernel_stack_watchdog.cc:198] Thread 2738 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 727ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:19.811866 2738 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.1:35409, user_credentials={real_user=slave}} blocked reactor thread for 1.00191e+06us
I20251212 08:53:19.813266 2589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.28"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:19.814682 2589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.28 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:19.828897 2795 tablet_service.cc:1505] Processing CreateTablet for tablet e8f092c5f152452695390c6d8ec19515 (DEFAULT_TABLE table=table.28 [id=113612d85174484bb74fbe621e339a66]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:19.829439 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8f092c5f152452695390c6d8ec19515. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:19.829689 2695 tablet_service.cc:1505] Processing CreateTablet for tablet e8f092c5f152452695390c6d8ec19515 (DEFAULT_TABLE table=table.28 [id=113612d85174484bb74fbe621e339a66]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:19.829939 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8f092c5f152452695390c6d8ec19515. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:19.831775 2694 tablet_service.cc:1505] Processing CreateTablet for tablet f2c37ac0543d4af8afa8efeb773dd11c (DEFAULT_TABLE table=table.28 [id=113612d85174484bb74fbe621e339a66]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:19.832122 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2c37ac0543d4af8afa8efeb773dd11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:19.828999 2876 tablet_service.cc:1505] Processing CreateTablet for tablet e8f092c5f152452695390c6d8ec19515 (DEFAULT_TABLE table=table.28 [id=113612d85174484bb74fbe621e339a66]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:19.832854 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e8f092c5f152452695390c6d8ec19515. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:19.834980 2873 tablet_service.cc:1505] Processing CreateTablet for tablet f2c37ac0543d4af8afa8efeb773dd11c (DEFAULT_TABLE table=table.28 [id=113612d85174484bb74fbe621e339a66]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:19.835307 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2c37ac0543d4af8afa8efeb773dd11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:19.836879 2793 tablet_service.cc:1505] Processing CreateTablet for tablet f2c37ac0543d4af8afa8efeb773dd11c (DEFAULT_TABLE table=table.28 [id=113612d85174484bb74fbe621e339a66]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:19.838750 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2c37ac0543d4af8afa8efeb773dd11c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:19.850375 3850 tablet_bootstrap.cc:492] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:19.850605 3851 tablet_bootstrap.cc:492] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:19.852938 3850 tablet_bootstrap.cc:654] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:19.853355 3852 tablet_bootstrap.cc:492] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:19.853964 3851 tablet_bootstrap.cc:654] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:19.856738 3852 tablet_bootstrap.cc:654] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:19.858511 3850 tablet_bootstrap.cc:492] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:19.858737 3850 ts_tablet_manager.cc:1403] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20251212 08:53:19.859678 3850 raft_consensus.cc:359] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:19.861234 3850 raft_consensus.cc:385] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:19.861337 3850 raft_consensus.cc:740] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:19.861617 3850 consensus_queue.cc:260] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:19.861166 3852 tablet_bootstrap.cc:492] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:19.862048 3852 ts_tablet_manager.cc:1403] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20251212 08:53:19.863276 3852 raft_consensus.cc:359] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:19.863512 3852 raft_consensus.cc:385] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:19.863631 3852 raft_consensus.cc:740] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:19.863869 3852 consensus_queue.cc:260] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:19.862557 3850 ts_tablet_manager.cc:1434] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:19.864375 3850 tablet_bootstrap.cc:492] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:19.866544 3852 ts_tablet_manager.cc:1434] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251212 08:53:19.866750 3850 tablet_bootstrap.cc:654] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:19.868160 3852 tablet_bootstrap.cc:492] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:19.869899 3852 tablet_bootstrap.cc:654] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:19.871867 3851 tablet_bootstrap.cc:492] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:19.875023 3851 ts_tablet_manager.cc:1403] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.000s
I20251212 08:53:19.875411 3850 tablet_bootstrap.cc:492] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:19.875536 3850 ts_tablet_manager.cc:1403] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20251212 08:53:19.875941 3851 raft_consensus.cc:359] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:19.876175 3851 raft_consensus.cc:385] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:19.876149 3850 raft_consensus.cc:359] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:19.876266 3851 raft_consensus.cc:740] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:19.876322 3850 raft_consensus.cc:385] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:19.876502 3850 raft_consensus.cc:740] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:19.876683 3850 consensus_queue.cc:260] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:19.877282 3850 ts_tablet_manager.cc:1434] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:19.876660 3851 consensus_queue.cc:260] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:19.878334 3851 ts_tablet_manager.cc:1434] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20251212 08:53:19.878624 3851 tablet_bootstrap.cc:492] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:19.881825 3851 tablet_bootstrap.cc:654] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:19.885438 3851 tablet_bootstrap.cc:492] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:19.886309 3851 ts_tablet_manager.cc:1403] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20251212 08:53:19.887460 3851 raft_consensus.cc:359] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:19.887754 3851 raft_consensus.cc:385] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:19.887861 3851 raft_consensus.cc:740] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:19.888131 3851 consensus_queue.cc:260] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:19.888902 3851 ts_tablet_manager.cc:1434] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:19.893090 3852 tablet_bootstrap.cc:492] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:19.893378 3852 ts_tablet_manager.cc:1403] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.010s
I20251212 08:53:19.894140 3852 raft_consensus.cc:359] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:19.894338 3852 raft_consensus.cc:385] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:19.894436 3852 raft_consensus.cc:740] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:19.894670 3852 consensus_queue.cc:260] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:19.895011 3752 raft_consensus.cc:493] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:19.895231 3852 ts_tablet_manager.cc:1434] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:19.895159 3752 raft_consensus.cc:515] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:19.895802 3752 leader_election.cc:290] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:19.896185 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2c37ac0543d4af8afa8efeb773dd11c" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:19.896361 2807 raft_consensus.cc:2468] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:19.896764 2667 leader_election.cc:304] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:19.896929 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2c37ac0543d4af8afa8efeb773dd11c" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:19.897009 3752 raft_consensus.cc:2804] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:19.897081 2887 raft_consensus.cc:2468] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:19.897069 3752 raft_consensus.cc:493] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:19.897265 3752 raft_consensus.cc:3060] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:19.900045 3752 raft_consensus.cc:515] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:19.900626 3752 leader_election.cc:290] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:19.901669 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2c37ac0543d4af8afa8efeb773dd11c" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:19.901852 2885 raft_consensus.cc:3060] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:19.901974 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f2c37ac0543d4af8afa8efeb773dd11c" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:19.902113 2807 raft_consensus.cc:3060] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:19.905272 2885 raft_consensus.cc:2468] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:19.905386 2807 raft_consensus.cc:2468] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:19.905705 2667 leader_election.cc:304] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:19.906041 3752 raft_consensus.cc:2804] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:19.907603 3752 raft_consensus.cc:697] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:19.907866 3752 consensus_queue.cc:237] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:19.913561 2586 catalog_manager.cc:5654] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:19.975714 3801 raft_consensus.cc:493] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:19.975896 3801 raft_consensus.cc:515] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:19.976603 3801 leader_election.cc:290] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:19.976974 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e8f092c5f152452695390c6d8ec19515" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:19.977083 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e8f092c5f152452695390c6d8ec19515" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:19.977150 2887 raft_consensus.cc:2468] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:19.977238 2705 raft_consensus.cc:2468] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:19.977624 2763 leader_election.cc:304] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:19.977965 3801 raft_consensus.cc:2804] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:19.978049 3801 raft_consensus.cc:493] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:19.978129 3801 raft_consensus.cc:3060] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:19.980504 3801 raft_consensus.cc:515] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:19.981060 3801 leader_election.cc:290] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:19.981329 2887 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e8f092c5f152452695390c6d8ec19515" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:19.981364 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e8f092c5f152452695390c6d8ec19515" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:19.981467 2887 raft_consensus.cc:3060] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:19.981513 2705 raft_consensus.cc:3060] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:19.983438 2887 raft_consensus.cc:2468] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:19.983882 2705 raft_consensus.cc:2468] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:19.983850 2765 leader_election.cc:304] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:19.984229 3801 raft_consensus.cc:2804] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:19.984407 3801 raft_consensus.cc:697] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:19.984628 3801 consensus_queue.cc:237] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:19.988729 2586 catalog_manager.cc:5654] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:20.032269 2887 raft_consensus.cc:1275] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:20.032727 2705 raft_consensus.cc:1275] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:20.032747 2807 raft_consensus.cc:1275] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:20.032747 2885 raft_consensus.cc:1275] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:20.033368 3801 consensus_queue.cc:1048] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:20.033933 3801 consensus_queue.cc:1048] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:20.034705 3859 consensus_queue.cc:1048] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:20.034888 3864 consensus_queue.cc:1048] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:20.382931 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 260ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:20.646716 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 527ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:20.910642 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 791ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:21.057209 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00102e+06us
I20251212 08:53:21.058022 2586 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.29"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:21.058789 2586 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.29 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:21.068765 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 63cc5ffd60084db6a67873d0f0b74c4a (DEFAULT_TABLE table=table.29 [id=afd0c3ee8ffa473093040b1712924a79]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:21.069211 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63cc5ffd60084db6a67873d0f0b74c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:21.069149 2694 tablet_service.cc:1505] Processing CreateTablet for tablet f03c86d8d28946a3b39af1b7d94a9226 (DEFAULT_TABLE table=table.29 [id=afd0c3ee8ffa473093040b1712924a79]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:21.069546 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f03c86d8d28946a3b39af1b7d94a9226. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:21.069931 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 63cc5ffd60084db6a67873d0f0b74c4a (DEFAULT_TABLE table=table.29 [id=afd0c3ee8ffa473093040b1712924a79]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:21.070243 2793 tablet_service.cc:1505] Processing CreateTablet for tablet f03c86d8d28946a3b39af1b7d94a9226 (DEFAULT_TABLE table=table.29 [id=afd0c3ee8ffa473093040b1712924a79]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:21.070592 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f03c86d8d28946a3b39af1b7d94a9226. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:21.070516 2876 tablet_service.cc:1505] Processing CreateTablet for tablet f03c86d8d28946a3b39af1b7d94a9226 (DEFAULT_TABLE table=table.29 [id=afd0c3ee8ffa473093040b1712924a79]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:21.075344 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 63cc5ffd60084db6a67873d0f0b74c4a (DEFAULT_TABLE table=table.29 [id=afd0c3ee8ffa473093040b1712924a79]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:21.075783 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63cc5ffd60084db6a67873d0f0b74c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:21.076210 3883 tablet_bootstrap.cc:492] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:21.070264 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 63cc5ffd60084db6a67873d0f0b74c4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:21.078017 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f03c86d8d28946a3b39af1b7d94a9226. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:21.078022 3883 tablet_bootstrap.cc:654] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:21.092474 3885 tablet_bootstrap.cc:492] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:21.094913 3885 tablet_bootstrap.cc:654] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:21.095259 3886 tablet_bootstrap.cc:492] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:21.097882 3886 tablet_bootstrap.cc:654] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:21.098414 3885 tablet_bootstrap.cc:492] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:21.098552 3885 ts_tablet_manager.cc:1403] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20251212 08:53:21.099268 3885 raft_consensus.cc:359] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:21.099419 3885 raft_consensus.cc:385] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:21.099494 3885 raft_consensus.cc:740] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:21.099761 3885 consensus_queue.cc:260] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:21.099931 3883 tablet_bootstrap.cc:492] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:21.100091 3883 ts_tablet_manager.cc:1403] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.004s
I20251212 08:53:21.100199 3885 ts_tablet_manager.cc:1434] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251212 08:53:21.100643 3885 tablet_bootstrap.cc:492] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:21.100793 3883 raft_consensus.cc:359] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:21.100948 3883 raft_consensus.cc:385] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:21.101039 3883 raft_consensus.cc:740] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:21.101230 3883 consensus_queue.cc:260] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:21.101991 3886 tablet_bootstrap.cc:492] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:21.102136 3886 ts_tablet_manager.cc:1403] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20251212 08:53:21.102250 3885 tablet_bootstrap.cc:654] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:21.102839 3886 raft_consensus.cc:359] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:21.103013 3886 raft_consensus.cc:385] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:21.103082 3886 raft_consensus.cc:740] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:21.103269 3886 consensus_queue.cc:260] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:21.103996 3886 ts_tablet_manager.cc:1434] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:21.105839 3886 tablet_bootstrap.cc:492] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:21.110296 3885 tablet_bootstrap.cc:492] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:21.110500 3885 ts_tablet_manager.cc:1403] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.006s
I20251212 08:53:21.111294 3886 tablet_bootstrap.cc:654] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:21.111735 3885 raft_consensus.cc:359] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:21.111922 3885 raft_consensus.cc:385] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:21.111985 3885 raft_consensus.cc:740] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:21.112200 3885 consensus_queue.cc:260] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:21.112718 3885 ts_tablet_manager.cc:1434] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:21.110296 3883 ts_tablet_manager.cc:1434] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20251212 08:53:21.113415 3883 tablet_bootstrap.cc:492] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:21.115083 3883 tablet_bootstrap.cc:654] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:21.115595 3886 tablet_bootstrap.cc:492] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:21.115715 3886 ts_tablet_manager.cc:1403] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20251212 08:53:21.116469 3886 raft_consensus.cc:359] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:21.116637 3886 raft_consensus.cc:385] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:21.116706 3886 raft_consensus.cc:740] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:21.116884 3886 consensus_queue.cc:260] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:21.118736 3886 ts_tablet_manager.cc:1434] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:53:21.119845 3883 tablet_bootstrap.cc:492] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:21.119946 3883 ts_tablet_manager.cc:1403] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.006s
I20251212 08:53:21.120553 3883 raft_consensus.cc:359] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:21.120682 3883 raft_consensus.cc:385] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:21.120749 3883 raft_consensus.cc:740] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:21.120944 3883 consensus_queue.cc:260] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:21.121446 3883 ts_tablet_manager.cc:1434] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:21.242177 3830 raft_consensus.cc:493] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:21.242372 3830 raft_consensus.cc:515] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:21.243100 3830 leader_election.cc:290] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:21.243515 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63cc5ffd60084db6a67873d0f0b74c4a" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:21.243475 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63cc5ffd60084db6a67873d0f0b74c4a" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:21.243702 2702 raft_consensus.cc:2468] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:21.243791 2805 raft_consensus.cc:2468] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:21.244203 2845 leader_election.cc:304] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:21.244518 3830 raft_consensus.cc:2804] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:21.244621 3830 raft_consensus.cc:493] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:21.244699 3830 raft_consensus.cc:3060] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:21.246990 3830 raft_consensus.cc:515] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:21.247501 3830 leader_election.cc:290] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:21.247831 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63cc5ffd60084db6a67873d0f0b74c4a" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:21.247957 2805 raft_consensus.cc:3060] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:21.249688 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "63cc5ffd60084db6a67873d0f0b74c4a" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:21.249835 2702 raft_consensus.cc:3060] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:21.250010 2805 raft_consensus.cc:2468] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:21.250380 2845 leader_election.cc:304] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:21.250615 3830 raft_consensus.cc:2804] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:21.250816 3830 raft_consensus.cc:697] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:21.251046 3830 consensus_queue.cc:237] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:21.252696 2702 raft_consensus.cc:2468] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:21.254956 2588 catalog_manager.cc:5654] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:21.329617 3859 raft_consensus.cc:493] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:21.329803 3859 raft_consensus.cc:515] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:21.330358 3859 leader_election.cc:290] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:21.330844 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f03c86d8d28946a3b39af1b7d94a9226" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:21.330852 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f03c86d8d28946a3b39af1b7d94a9226" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:21.331056 2885 raft_consensus.cc:2468] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:21.331056 2807 raft_consensus.cc:2468] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:21.331429 2667 leader_election.cc:304] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:21.331643 3859 raft_consensus.cc:2804] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:21.331730 3859 raft_consensus.cc:493] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:21.331800 3859 raft_consensus.cc:3060] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:21.334391 3859 raft_consensus.cc:515] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:21.334985 3859 leader_election.cc:290] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:21.335402 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f03c86d8d28946a3b39af1b7d94a9226" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:21.335547 2885 raft_consensus.cc:3060] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:21.335624 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f03c86d8d28946a3b39af1b7d94a9226" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:21.335790 2807 raft_consensus.cc:3060] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:21.338752 2885 raft_consensus.cc:2468] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:21.338876 2807 raft_consensus.cc:2468] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:21.339109 2667 leader_election.cc:304] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:21.339355 3859 raft_consensus.cc:2804] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:21.339499 3859 raft_consensus.cc:697] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:21.339767 3859 consensus_queue.cc:237] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:21.354254 2589 catalog_manager.cc:5654] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:21.385744 2807 raft_consensus.cc:1275] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:21.385869 2885 raft_consensus.cc:1275] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:21.386554 2805 raft_consensus.cc:1275] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:21.386505 3864 consensus_queue.cc:1048] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:21.386797 3893 consensus_queue.cc:1048] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:21.387086 3830 consensus_queue.cc:1048] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:21.387555 2704 raft_consensus.cc:1275] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:21.388135 3830 consensus_queue.cc:1048] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:21.406774 3898 mvcc.cc:204] Tried to move back new op lower bound from 7231609247272685568 to 7231609246726766592. Current Snapshot: MvccSnapshot[applied={T|T < 7231609247272685568 or (T in {7231609247272685568})}]
W20251212 08:53:21.582978 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 101ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:21.844516 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 372ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:22.106387 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 634ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:22.369748 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 895ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:22.412093 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.3:33005, user_credentials={real_user=slave}} blocked reactor thread for 1.00107e+06us
I20251212 08:53:22.413316 2589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.30"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:22.414409 2589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.30 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:22.425583 2695 tablet_service.cc:1505] Processing CreateTablet for tablet fc7584a5fb014c039b1a3f4197cd017a (DEFAULT_TABLE table=table.30 [id=0f61254190664099ae4a093838ebeb30]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:22.426074 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc7584a5fb014c039b1a3f4197cd017a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:22.426127 2694 tablet_service.cc:1505] Processing CreateTablet for tablet c537842e2a634782afb1ea70568d698c (DEFAULT_TABLE table=table.30 [id=0f61254190664099ae4a093838ebeb30]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:22.426450 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c537842e2a634782afb1ea70568d698c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:22.425628 2876 tablet_service.cc:1505] Processing CreateTablet for tablet fc7584a5fb014c039b1a3f4197cd017a (DEFAULT_TABLE table=table.30 [id=0f61254190664099ae4a093838ebeb30]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:22.426930 2876 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc7584a5fb014c039b1a3f4197cd017a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:22.432688 2795 tablet_service.cc:1505] Processing CreateTablet for tablet fc7584a5fb014c039b1a3f4197cd017a (DEFAULT_TABLE table=table.30 [id=0f61254190664099ae4a093838ebeb30]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:22.433066 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fc7584a5fb014c039b1a3f4197cd017a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:22.434283 2873 tablet_service.cc:1505] Processing CreateTablet for tablet c537842e2a634782afb1ea70568d698c (DEFAULT_TABLE table=table.30 [id=0f61254190664099ae4a093838ebeb30]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:22.434644 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c537842e2a634782afb1ea70568d698c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:22.436692 2793 tablet_service.cc:1505] Processing CreateTablet for tablet c537842e2a634782afb1ea70568d698c (DEFAULT_TABLE table=table.30 [id=0f61254190664099ae4a093838ebeb30]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:22.437047 2793 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c537842e2a634782afb1ea70568d698c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:22.444977 3915 tablet_bootstrap.cc:492] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:22.447315 3915 tablet_bootstrap.cc:654] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:22.454525 3917 tablet_bootstrap.cc:492] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:22.455778 3916 tablet_bootstrap.cc:492] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:22.458525 3916 tablet_bootstrap.cc:654] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:22.459475 3917 tablet_bootstrap.cc:654] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:22.460883 3915 tablet_bootstrap.cc:492] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:22.461089 3915 ts_tablet_manager.cc:1403] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.009s
I20251212 08:53:22.461445 3916 tablet_bootstrap.cc:492] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:22.461601 3916 ts_tablet_manager.cc:1403] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251212 08:53:22.461885 3915 raft_consensus.cc:359] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:22.462092 3915 raft_consensus.cc:385] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:22.462180 3915 raft_consensus.cc:740] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:22.462286 3916 raft_consensus.cc:359] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:22.462450 3916 raft_consensus.cc:385] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:22.462401 3915 consensus_queue.cc:260] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:22.462509 3916 raft_consensus.cc:740] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:22.462700 3916 consensus_queue.cc:260] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:22.462956 3915 ts_tablet_manager.cc:1434] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:22.463124 3916 ts_tablet_manager.cc:1434] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:22.463260 3915 tablet_bootstrap.cc:492] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:22.463351 3916 tablet_bootstrap.cc:492] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:22.464954 3917 tablet_bootstrap.cc:492] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:22.465075 3917 ts_tablet_manager.cc:1403] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.001s
I20251212 08:53:22.465150 3915 tablet_bootstrap.cc:654] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:22.465150 3916 tablet_bootstrap.cc:654] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:22.465845 3917 raft_consensus.cc:359] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:22.466002 3917 raft_consensus.cc:385] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:22.466051 3917 raft_consensus.cc:740] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:22.466216 3917 consensus_queue.cc:260] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:22.466651 3917 ts_tablet_manager.cc:1434] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:53:22.466892 3917 tablet_bootstrap.cc:492] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:22.467718 3877 raft_consensus.cc:493] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:22.467836 3877 raft_consensus.cc:515] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:22.468489 3877 leader_election.cc:290] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:22.468956 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc7584a5fb014c039b1a3f4197cd017a" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
W20251212 08:53:22.469522 2843 leader_election.cc:343] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Illegal state: must be running to vote when last-logged opid is not known
I20251212 08:53:22.471000 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc7584a5fb014c039b1a3f4197cd017a" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:22.471161 2807 raft_consensus.cc:2468] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:22.472198 2845 leader_election.cc:304] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters: 7cab108f9abb4e1c973338419932d96c
I20251212 08:53:22.472766 3877 raft_consensus.cc:2804] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:22.472844 3877 raft_consensus.cc:493] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:22.472905 3877 raft_consensus.cc:3060] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:22.473279 3915 tablet_bootstrap.cc:492] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:22.473279 3916 tablet_bootstrap.cc:492] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:22.473416 3915 ts_tablet_manager.cc:1403] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.008s
I20251212 08:53:22.473428 3916 ts_tablet_manager.cc:1403] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20251212 08:53:22.474216 3915 raft_consensus.cc:359] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:22.474372 3915 raft_consensus.cc:385] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:22.474323 3916 raft_consensus.cc:359] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:22.474418 3915 raft_consensus.cc:740] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:22.474448 3916 raft_consensus.cc:385] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:22.474494 3916 raft_consensus.cc:740] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:22.474634 3916 consensus_queue.cc:260] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:22.474609 3915 consensus_queue.cc:260] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:22.475111 3916 ts_tablet_manager.cc:1434] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251212 08:53:22.475195 3915 ts_tablet_manager.cc:1434] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20251212 08:53:22.475752 3877 raft_consensus.cc:515] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:22.477408 3917 tablet_bootstrap.cc:654] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:22.479375 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc7584a5fb014c039b1a3f4197cd017a" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:22.479558 2807 raft_consensus.cc:3060] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:22.479645 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fc7584a5fb014c039b1a3f4197cd017a" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:22.479774 2702 raft_consensus.cc:3060] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:22.482156 3917 tablet_bootstrap.cc:492] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:22.482318 2702 raft_consensus.cc:2468] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:22.479552 3877 leader_election.cc:290] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409)
I20251212 08:53:22.482409 2807 raft_consensus.cc:2468] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:22.482816 2843 leader_election.cc:304] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:22.483158 3877 raft_consensus.cc:2804] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:22.483299 3877 raft_consensus.cc:697] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:22.483551 3877 consensus_queue.cc:237] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:22.484143 3917 ts_tablet_manager.cc:1403] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.004s
I20251212 08:53:22.485040 3917 raft_consensus.cc:359] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:22.485302 3917 raft_consensus.cc:385] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:22.486949 3917 raft_consensus.cc:740] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:22.487275 3917 consensus_queue.cc:260] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:22.487617 2586 catalog_manager.cc:5654] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:22.488754 3917 ts_tablet_manager.cc:1434] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:22.672883 3910 raft_consensus.cc:493] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:22.673025 3910 raft_consensus.cc:515] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:22.673554 3910 leader_election.cc:290] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:22.673825 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c537842e2a634782afb1ea70568d698c" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:22.674006 2884 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c537842e2a634782afb1ea70568d698c" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:22.674166 2884 raft_consensus.cc:2468] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:22.674005 2702 raft_consensus.cc:2468] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 0.
I20251212 08:53:22.674575 2765 leader_election.cc:304] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:22.674790 3910 raft_consensus.cc:2804] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:22.674854 3910 raft_consensus.cc:493] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:22.674938 3910 raft_consensus.cc:3060] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:22.676843 3910 raft_consensus.cc:515] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:22.677354 3910 leader_election.cc:290] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:22.677790 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c537842e2a634782afb1ea70568d698c" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:22.677935 2885 raft_consensus.cc:3060] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:22.677943 2702 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c537842e2a634782afb1ea70568d698c" candidate_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:22.678110 2702 raft_consensus.cc:3060] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:22.680193 2885 raft_consensus.cc:2468] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:22.680259 2702 raft_consensus.cc:2468] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7aad0f0d166f4050b76cd4d94dbbbc78 in term 1.
I20251212 08:53:22.680603 2765 leader_election.cc:304] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7aad0f0d166f4050b76cd4d94dbbbc78; no voters:
I20251212 08:53:22.680810 3910 raft_consensus.cc:2804] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:22.680992 3910 raft_consensus.cc:697] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Becoming Leader. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Running, Role: LEADER
I20251212 08:53:22.681231 3910 consensus_queue.cc:237] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:22.685699 2589 catalog_manager.cc:5654] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:22.745585 2885 raft_consensus.cc:1275] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:22.745656 2702 raft_consensus.cc:1275] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 7aad0f0d166f4050b76cd4d94dbbbc78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:22.745869 2807 raft_consensus.cc:1275] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:22.746268 2704 raft_consensus.cc:1275] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:22.746354 3910 consensus_queue.cc:1048] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:22.746476 3830 consensus_queue.cc:1048] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:22.746511 3878 consensus_queue.cc:1048] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:22.748404 3830 consensus_queue.cc:1048] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:23.038759 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 195ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:23.302870 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 463ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:23.565603 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 727ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:23.776592 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.2:41879, user_credentials={real_user=slave}} blocked reactor thread for 1.00105e+06us
I20251212 08:53:23.777864 2589 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59396:
name: "table.31"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\001\000\000\000""\004\001\000\001\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251212 08:53:23.778977 2589 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table table.31 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:53:23.794013 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 29970481f9134918855691767fe1af96 (DEFAULT_TABLE table=table.31 [id=0cae36934e244bd4b2f06c43874995b8]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:23.794548 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29970481f9134918855691767fe1af96. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:23.804551 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 29970481f9134918855691767fe1af96 (DEFAULT_TABLE table=table.31 [id=0cae36934e244bd4b2f06c43874995b8]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:23.805006 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29970481f9134918855691767fe1af96. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:23.805199 2694 tablet_service.cc:1505] Processing CreateTablet for tablet 29970481f9134918855691767fe1af96 (DEFAULT_TABLE table=table.31 [id=0cae36934e244bd4b2f06c43874995b8]), partition=RANGE (key) PARTITION VALUES < 1
I20251212 08:53:23.805699 2694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29970481f9134918855691767fe1af96. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:23.819448 2695 tablet_service.cc:1505] Processing CreateTablet for tablet 84832bfc9dd444a39aff9800980365ca (DEFAULT_TABLE table=table.31 [id=0cae36934e244bd4b2f06c43874995b8]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:23.820108 2695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84832bfc9dd444a39aff9800980365ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:23.821823 3946 tablet_bootstrap.cc:492] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:23.822038 3948 tablet_bootstrap.cc:492] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:23.823798 3948 tablet_bootstrap.cc:654] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:23.824064 3946 tablet_bootstrap.cc:654] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:23.824754 3947 tablet_bootstrap.cc:492] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:23.827332 3947 tablet_bootstrap.cc:654] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:23.830670 2873 tablet_service.cc:1505] Processing CreateTablet for tablet 84832bfc9dd444a39aff9800980365ca (DEFAULT_TABLE table=table.31 [id=0cae36934e244bd4b2f06c43874995b8]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:23.830837 2795 tablet_service.cc:1505] Processing CreateTablet for tablet 84832bfc9dd444a39aff9800980365ca (DEFAULT_TABLE table=table.31 [id=0cae36934e244bd4b2f06c43874995b8]), partition=RANGE (key) PARTITION 1 <= VALUES
I20251212 08:53:23.831040 2873 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84832bfc9dd444a39aff9800980365ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:23.831240 2795 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 84832bfc9dd444a39aff9800980365ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:23.842388 3947 tablet_bootstrap.cc:492] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:23.842563 3947 ts_tablet_manager.cc:1403] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.018s user 0.001s sys 0.003s
I20251212 08:53:23.843511 3947 raft_consensus.cc:359] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:23.843868 3947 raft_consensus.cc:385] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:23.843981 3947 raft_consensus.cc:740] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:23.844273 3947 consensus_queue.cc:260] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:23.844988 3947 ts_tablet_manager.cc:1434] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20251212 08:53:23.845358 3947 tablet_bootstrap.cc:492] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78: Bootstrap starting.
I20251212 08:53:23.846796 3946 tablet_bootstrap.cc:492] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:23.846975 3946 ts_tablet_manager.cc:1403] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.006s
I20251212 08:53:23.847059 3948 tablet_bootstrap.cc:492] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:23.847189 3948 ts_tablet_manager.cc:1403] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.003s
I20251212 08:53:23.847358 3947 tablet_bootstrap.cc:654] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:23.847992 3946 raft_consensus.cc:359] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:23.848497 3948 raft_consensus.cc:359] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:23.848618 3946 raft_consensus.cc:385] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:23.848729 3948 raft_consensus.cc:385] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:23.848771 3946 raft_consensus.cc:740] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:23.848795 3948 raft_consensus.cc:740] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:23.849120 3946 consensus_queue.cc:260] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:23.849337 3947 tablet_bootstrap.cc:492] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78: No bootstrap required, opened a new log
I20251212 08:53:23.849117 3948 consensus_queue.cc:260] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:23.849464 3947 ts_tablet_manager.cc:1403] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20251212 08:53:23.849869 3948 ts_tablet_manager.cc:1434] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:23.849876 3946 ts_tablet_manager.cc:1434] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251212 08:53:23.850123 3948 tablet_bootstrap.cc:492] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c: Bootstrap starting.
I20251212 08:53:23.850183 3947 raft_consensus.cc:359] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:23.850282 3946 tablet_bootstrap.cc:492] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132: Bootstrap starting.
I20251212 08:53:23.850343 3947 raft_consensus.cc:385] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:23.850445 3947 raft_consensus.cc:740] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aad0f0d166f4050b76cd4d94dbbbc78, State: Initialized, Role: FOLLOWER
I20251212 08:53:23.850664 3947 consensus_queue.cc:260] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:23.851166 3947 ts_tablet_manager.cc:1434] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:23.851970 3948 tablet_bootstrap.cc:654] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:23.852030 3946 tablet_bootstrap.cc:654] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:23.857679 3946 tablet_bootstrap.cc:492] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132: No bootstrap required, opened a new log
I20251212 08:53:23.857836 3946 ts_tablet_manager.cc:1403] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20251212 08:53:23.858507 3946 raft_consensus.cc:359] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:23.858647 3946 raft_consensus.cc:385] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:23.860102 3946 raft_consensus.cc:740] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Initialized, Role: FOLLOWER
I20251212 08:53:23.860401 3946 consensus_queue.cc:260] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:23.860910 3946 ts_tablet_manager.cc:1434] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:23.869951 3948 tablet_bootstrap.cc:492] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c: No bootstrap required, opened a new log
I20251212 08:53:23.870110 3948 ts_tablet_manager.cc:1403] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.013s
I20251212 08:53:23.871238 3948 raft_consensus.cc:359] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:23.871430 3948 raft_consensus.cc:385] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:23.871486 3948 raft_consensus.cc:740] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Initialized, Role: FOLLOWER
I20251212 08:53:23.871944 3948 consensus_queue.cc:260] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:23.872630 3948 ts_tablet_manager.cc:1434] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:23.993831 3893 raft_consensus.cc:493] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:23.994006 3893 raft_consensus.cc:515] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:23.994546 3893 leader_election.cc:290] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:23.994952 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29970481f9134918855691767fe1af96" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:23.995007 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29970481f9134918855691767fe1af96" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132" is_pre_election: true
I20251212 08:53:23.995162 2807 raft_consensus.cc:2468] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:23.995162 2885 raft_consensus.cc:2468] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 0.
I20251212 08:53:23.995607 2667 leader_election.cc:304] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:23.995808 3893 raft_consensus.cc:2804] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:23.995893 3893 raft_consensus.cc:493] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:23.995961 3893 raft_consensus.cc:3060] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:23.998845 3893 raft_consensus.cc:515] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:23.999437 3893 leader_election.cc:290] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Requested vote from peers 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879), 2d41c990aa9f453e96ec315fab508132 (127.0.208.3:33005)
I20251212 08:53:23.999776 2885 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29970481f9134918855691767fe1af96" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d41c990aa9f453e96ec315fab508132"
I20251212 08:53:23.999776 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "29970481f9134918855691767fe1af96" candidate_uuid: "7cab108f9abb4e1c973338419932d96c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:23.999989 2885 raft_consensus.cc:3060] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:24.000036 2807 raft_consensus.cc:3060] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:24.002810 2885 raft_consensus.cc:2468] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:24.002810 2807 raft_consensus.cc:2468] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cab108f9abb4e1c973338419932d96c in term 1.
I20251212 08:53:24.003278 2667 leader_election.cc:304] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7aad0f0d166f4050b76cd4d94dbbbc78, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:24.003593 3893 raft_consensus.cc:2804] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:24.003726 3893 raft_consensus.cc:697] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Becoming Leader. State: Replica: 7cab108f9abb4e1c973338419932d96c, State: Running, Role: LEADER
I20251212 08:53:24.004061 3893 consensus_queue.cc:237] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } }
I20251212 08:53:24.007570 2585 catalog_manager.cc:5654] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cab108f9abb4e1c973338419932d96c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "7cab108f9abb4e1c973338419932d96c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:24.021521 3877 raft_consensus.cc:493] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:24.021710 3877 raft_consensus.cc:515] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:24.022382 3877 leader_election.cc:290] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:24.022650 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84832bfc9dd444a39aff9800980365ca" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c" is_pre_election: true
I20251212 08:53:24.022826 2705 raft_consensus.cc:2468] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:24.022995 2805 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84832bfc9dd444a39aff9800980365ca" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" is_pre_election: true
I20251212 08:53:24.023190 2805 raft_consensus.cc:2468] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 0.
I20251212 08:53:24.023216 2843 leader_election.cc:304] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:24.023627 3877 raft_consensus.cc:2804] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:24.023711 3877 raft_consensus.cc:493] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:24.023789 3877 raft_consensus.cc:3060] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:24.026571 3877 raft_consensus.cc:515] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:24.027034 3877 leader_election.cc:290] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Requested vote from peers 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409), 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879)
I20251212 08:53:24.027424 2705 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84832bfc9dd444a39aff9800980365ca" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cab108f9abb4e1c973338419932d96c"
I20251212 08:53:24.027424 2807 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "84832bfc9dd444a39aff9800980365ca" candidate_uuid: "2d41c990aa9f453e96ec315fab508132" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78"
I20251212 08:53:24.027599 2705 raft_consensus.cc:3060] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:24.027627 2807 raft_consensus.cc:3060] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:24.029502 2705 raft_consensus.cc:2468] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:24.029587 2807 raft_consensus.cc:2468] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d41c990aa9f453e96ec315fab508132 in term 1.
I20251212 08:53:24.029878 2843 leader_election.cc:304] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d41c990aa9f453e96ec315fab508132, 7cab108f9abb4e1c973338419932d96c; no voters:
I20251212 08:53:24.030107 3877 raft_consensus.cc:2804] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:24.030215 3877 raft_consensus.cc:697] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Becoming Leader. State: Replica: 2d41c990aa9f453e96ec315fab508132, State: Running, Role: LEADER
I20251212 08:53:24.030438 3877 consensus_queue.cc:237] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } }
I20251212 08:53:24.034567 2588 catalog_manager.cc:5654] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d41c990aa9f453e96ec315fab508132 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "2d41c990aa9f453e96ec315fab508132" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:24.061194 2807 raft_consensus.cc:1275] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:24.061193 2705 raft_consensus.cc:1275] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Refusing update from remote peer 2d41c990aa9f453e96ec315fab508132: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:24.061728 2805 raft_consensus.cc:1275] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:24.061806 3877 consensus_queue.cc:1048] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:24.061902 2885 raft_consensus.cc:1275] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Refusing update from remote peer 7cab108f9abb4e1c973338419932d96c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:24.062491 3897 consensus_queue.cc:1048] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d41c990aa9f453e96ec315fab508132" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 33005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:24.062638 3893 consensus_queue.cc:1048] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aad0f0d166f4050b76cd4d94dbbbc78" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 41879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:24.065778 3941 consensus_queue.cc:1048] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cab108f9abb4e1c973338419932d96c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251212 08:53:24.330895 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 189ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:24.596416 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 453ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:24.863106 836 kernel_stack_watchdog.cc:198] Thread 2740 stuck at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:307 for 718ms:
Kernel stack:
(could not read kernel stack)
User stack:
@ 0x7f55c5109980 (unknown) at ??:0
@ 0x7f55c5108d50 __GI___nanosleep at ??:0
@ 0x7f55c10a6e7b base::SleepForNanoseconds() at ??:0
@ 0x7f55c21bd986 kudu::SleepFor() at ??:0
@ 0xa2b124 kudu::client::ClientTest_TestAutoFlushBackgroundAndErrorCollector_Test::TestBody()::CustomErrorCollector::AddError() at /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/client-test.cc:4513
@ 0x7f55d0280a23 kudu::client::internal::Batcher::ProcessWriteResponse() at ??:0
@ 0x7f55d027f526 kudu::client::internal::WriteRpc::Finish() at ??:0
@ 0x7f55d0290795 kudu::rpc::RetriableRpc<>::SendRpcCb() at ??:0
@ 0x7f55d02ab031 _ZZN4kudu3rpc12RetriableRpcINS_6client8internal18RemoteTabletServerENS_7tserver14WriteRequestPBENS5_15WriteResponsePBEE14ReplicaFoundCbERKNS_6StatusEPS4_ENKUlvE_clEv at ??:0
@ 0x7f55d02aad7e _ZNSt17_Function_handlerIFvvEZN4kudu3rpc12RetriableRpcINS1_6client8internal18RemoteTabletServerENS1_7tserver14WriteRequestPBENS7_15WriteResponsePBEE14ReplicaFoundCbERKNS1_6StatusEPS6_EUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0xc0c00d std::function<>::operator()() at /usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:706
@ 0x7f55c8c82f20 kudu::rpc::OutboundCall::CallCallback() at ??:0
@ 0x7f55c8c8346c kudu::rpc::OutboundCall::SetResponse() at ??:0
@ 0x7f55c8c1e96a kudu::rpc::Connection::HandleCallResponse() at ??:0
@ 0x7f55c8c1d326 kudu::rpc::Connection::ReadHandler() at ??:0
@ 0x7f55c8c2f7f5 ev::base<>::method_thunk<>() at ??:0
W20251212 08:53:25.079203 2740 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.0.208.3:33005, user_credentials={real_user=slave}} blocked reactor thread for 1.00121e+06us
I20251212 08:53:25.080927 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
W20251212 08:53:25.095248 2843 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 2 similar messages]
I20251212 08:53:25.095969 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:25.096562 832 tablet_replica.cc:333] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.096827 832 raft_consensus.cc:2243] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.097354 832 raft_consensus.cc:2272] T 29970481f9134918855691767fe1af96 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.099493 832 tablet_replica.cc:333] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
W20251212 08:53:25.099545 2763 consensus_peers.cc:597] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.099677 832 raft_consensus.cc:2243] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.099859 832 raft_consensus.cc:2272] T c537842e2a634782afb1ea70568d698c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.100464 832 tablet_replica.cc:333] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.100621 832 raft_consensus.cc:2243] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.100795 832 raft_consensus.cc:2272] T 63cc5ffd60084db6a67873d0f0b74c4a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.101404 832 tablet_replica.cc:333] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.101613 832 raft_consensus.cc:2243] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.101907 832 raft_consensus.cc:2272] T f2c37ac0543d4af8afa8efeb773dd11c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.102507 832 tablet_replica.cc:333] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.102705 832 raft_consensus.cc:2243] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.102875 832 raft_consensus.cc:2272] T e8f092c5f152452695390c6d8ec19515 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.103464 832 tablet_replica.cc:333] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.103657 832 raft_consensus.cc:2243] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.103969 832 raft_consensus.cc:2272] T c57c808afa054124984ee62dcaa4c22e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.104736 832 tablet_replica.cc:333] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
W20251212 08:53:25.104802 2843 consensus_peers.cc:597] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.104996 832 raft_consensus.cc:2243] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.105191 832 raft_consensus.cc:2272] T c0aa78ae4efc4856a0262730d672874f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.105792 832 tablet_replica.cc:333] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.106030 832 raft_consensus.cc:2243] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.106215 832 raft_consensus.cc:2272] T bc958968094c4914ae8e620f1cfb94fa P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.106817 832 tablet_replica.cc:333] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.107019 832 raft_consensus.cc:2243] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.107205 832 raft_consensus.cc:2272] T e66ba33ab78f4604b175eef3351f38c1 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.107796 832 tablet_replica.cc:333] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.107997 832 raft_consensus.cc:2243] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.108337 832 raft_consensus.cc:2272] T 4802e69098d74c6eba89068ae8a73106 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.108987 832 tablet_replica.cc:333] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.109184 832 raft_consensus.cc:2243] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.109508 832 raft_consensus.cc:2272] T d7e15336531948be810a2944afa986a4 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.110141 832 tablet_replica.cc:333] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.110374 832 raft_consensus.cc:2243] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.110669 832 raft_consensus.cc:2272] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.111281 832 tablet_replica.cc:333] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.111553 832 raft_consensus.cc:2243] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.111908 832 raft_consensus.cc:2272] T de0691c0e98f4700b784a2981f741b5a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.112618 832 tablet_replica.cc:333] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.115022 832 raft_consensus.cc:2243] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.115253 832 raft_consensus.cc:2272] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.115953 832 tablet_replica.cc:333] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.116154 832 raft_consensus.cc:2243] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.116487 832 raft_consensus.cc:2272] T f03c86d8d28946a3b39af1b7d94a9226 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.117075 832 tablet_replica.cc:333] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.117349 832 raft_consensus.cc:2243] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.117573 832 raft_consensus.cc:2272] T e389399dd80a4284b150fc473d1928e8 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.118214 832 tablet_replica.cc:333] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.118441 832 raft_consensus.cc:2243] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.118654 832 raft_consensus.cc:2272] T db9329c939f846f48bd173a350bef36d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.119238 832 tablet_replica.cc:333] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.119470 832 raft_consensus.cc:2243] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.119652 832 raft_consensus.cc:2272] T 9bbebd39db8945459705e59f8670eb3f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.120337 832 tablet_replica.cc:333] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.120606 832 raft_consensus.cc:2243] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.120857 832 raft_consensus.cc:2272] T 84832bfc9dd444a39aff9800980365ca P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.121495 832 tablet_replica.cc:333] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.121703 832 raft_consensus.cc:2243] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.121901 832 raft_consensus.cc:2272] T 2fd487ada3bd4c41b299172d81e63b7e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.122555 832 tablet_replica.cc:333] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.122820 832 raft_consensus.cc:2243] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.123062 832 raft_consensus.cc:2272] T 62b27193ad064f6e87f51a3e3a8756bc P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.123737 832 tablet_replica.cc:333] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.123935 832 raft_consensus.cc:2243] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.124244 832 raft_consensus.cc:2272] T d84415e201b34feb9991ac04c8355802 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.124931 832 tablet_replica.cc:333] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.125190 832 raft_consensus.cc:2243] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.125432 832 raft_consensus.cc:2272] T 82b6f18e15f941929f188b2b490718ca P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.126106 832 tablet_replica.cc:333] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.126318 832 raft_consensus.cc:2243] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.126508 832 raft_consensus.cc:2272] T 5a5ad45938984dbfbb2349fa3c85fccb P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.128683 832 tablet_replica.cc:333] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.128852 832 raft_consensus.cc:2243] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.129259 832 raft_consensus.cc:2272] T 6b147d7c9a3c44808ebae11ec97725e0 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.129673 832 tablet_replica.cc:333] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.129832 832 raft_consensus.cc:2243] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.129977 832 raft_consensus.cc:2272] T 92719eed5a0448f2a99ac8897be6c266 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.130447 832 tablet_replica.cc:333] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.130609 832 raft_consensus.cc:2243] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.130877 832 raft_consensus.cc:2272] T 888c406ad456495cb6bb53abac986a94 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.131353 832 tablet_replica.cc:333] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.131490 832 raft_consensus.cc:2243] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.131629 832 raft_consensus.cc:2272] T d4841a0029994cc3a2d02e5889d34958 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.132028 832 tablet_replica.cc:333] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.132169 832 raft_consensus.cc:2243] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.132342 832 raft_consensus.cc:2272] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.132738 832 tablet_replica.cc:333] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.132870 832 raft_consensus.cc:2243] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.133008 832 raft_consensus.cc:2272] T 8f66b4c1169b4621838e302888e31047 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.133458 832 tablet_replica.cc:333] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.133634 832 raft_consensus.cc:2243] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.133802 832 raft_consensus.cc:2272] T 0ebda846433246b0af04f3126d32f4ce P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.134347 832 tablet_replica.cc:333] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.134505 832 raft_consensus.cc:2243] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
W20251212 08:53:25.134984 2763 consensus_peers.cc:597] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.136170 2763 consensus_peers.cc:597] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.146813 2763 consensus_peers.cc:597] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.149780 2763 consensus_peers.cc:597] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.157590 832 raft_consensus.cc:2272] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.158192 832 tablet_replica.cc:333] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.158376 832 raft_consensus.cc:2243] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.158525 832 raft_consensus.cc:2272] T 47b760e1902f4f5f84cdb54634e6da97 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.158941 832 tablet_replica.cc:333] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.159076 832 raft_consensus.cc:2243] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.159349 832 raft_consensus.cc:2272] T c747caeb72d148f1a70b0ce32ea9f7ff P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.159821 832 tablet_replica.cc:333] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.159991 832 raft_consensus.cc:2243] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.160125 832 raft_consensus.cc:2272] T 0fd6c689a671492194b6e799e903bfe0 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.160686 832 tablet_replica.cc:333] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.160841 832 raft_consensus.cc:2243] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.161098 832 raft_consensus.cc:2272] T c40e9cbf2b184cb7be83ed003ecadfde P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.161512 832 tablet_replica.cc:333] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.161657 832 raft_consensus.cc:2243] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.161820 832 raft_consensus.cc:2272] T a295a2fa31fe405da6380ccfd082338b P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.162175 832 tablet_replica.cc:333] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.162297 832 raft_consensus.cc:2243] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.162410 832 raft_consensus.cc:2272] T 0e1c7c8118304721a5c37409bb220623 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.162752 832 tablet_replica.cc:333] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.162851 832 raft_consensus.cc:2243] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.163060 832 raft_consensus.cc:2272] T d897186bf4c3441caefa0c2a5d83fc4c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.163513 832 tablet_replica.cc:333] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.163663 832 raft_consensus.cc:2243] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.163880 832 raft_consensus.cc:2272] T 50e34e7a14274ee9ad9d640eb15a6909 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.164389 832 tablet_replica.cc:333] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.164536 832 raft_consensus.cc:2243] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.164686 832 raft_consensus.cc:2272] T 7d16ad80c41b41e2accecd66ca20e04a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.165073 832 tablet_replica.cc:333] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.165208 832 raft_consensus.cc:2243] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.165417 832 raft_consensus.cc:2272] T a9cc0c9708724db590f46d357c6a3639 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.165807 832 tablet_replica.cc:333] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.165936 832 raft_consensus.cc:2243] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.166074 832 raft_consensus.cc:2272] T dfc6271b43244577a6b05d092920f236 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.166452 832 tablet_replica.cc:333] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.166591 832 raft_consensus.cc:2243] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.166728 832 raft_consensus.cc:2272] T 337002230a3949a1af139ac4ac693f81 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.167116 832 tablet_replica.cc:333] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.167263 832 raft_consensus.cc:2243] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.167433 832 raft_consensus.cc:2272] T ff97ced42b1543b2a2b41d4106de4e5c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.167853 832 tablet_replica.cc:333] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.167996 832 raft_consensus.cc:2243] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.168354 832 raft_consensus.cc:2272] T c3c476911f1449d299b316d4d5730c71 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.168764 832 tablet_replica.cc:333] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.168900 832 raft_consensus.cc:2243] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.169019 832 raft_consensus.cc:2272] T e2f4303ab1754ce58f47f39d80e9c38c P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.169433 832 tablet_replica.cc:333] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.169597 832 raft_consensus.cc:2243] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.169749 832 raft_consensus.cc:2272] T 31333c1e9c574d0bb2f38ca25d972d70 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.170217 832 tablet_replica.cc:333] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.170385 832 raft_consensus.cc:2243] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.170643 832 raft_consensus.cc:2272] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.171130 832 tablet_replica.cc:333] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.171272 832 raft_consensus.cc:2243] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.171455 832 raft_consensus.cc:2272] T bcd3ea4b4e7341b59b92ce737d89af33 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.171865 832 tablet_replica.cc:333] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.171990 832 raft_consensus.cc:2243] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.172197 832 raft_consensus.cc:2272] T 0f2534bb7ce049a19359b3c97a383cec P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.172605 832 tablet_replica.cc:333] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.172735 832 raft_consensus.cc:2243] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.172936 832 raft_consensus.cc:2272] T 38ebd93ba8814c59b510e570877bca91 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.173426 832 tablet_replica.cc:333] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.173563 832 raft_consensus.cc:2243] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.173708 832 raft_consensus.cc:2272] T 85bd9f82cab946bfa260338f237a589d P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.174098 832 tablet_replica.cc:333] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.174225 832 raft_consensus.cc:2243] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.174367 832 raft_consensus.cc:2272] T 8638463f714e4dddbf5cec9d89890e4e P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
W20251212 08:53:25.174482 2763 consensus_peers.cc:597] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.174770 832 tablet_replica.cc:333] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.174888 832 raft_consensus.cc:2243] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.175019 832 raft_consensus.cc:2272] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.175431 832 tablet_replica.cc:333] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.175535 832 raft_consensus.cc:2243] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.175709 832 raft_consensus.cc:2272] T 6d67e78322704917859cd6c6e62b6964 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.176101 832 tablet_replica.cc:333] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.176201 832 raft_consensus.cc:2243] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.176364 832 raft_consensus.cc:2272] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.176740 832 tablet_replica.cc:333] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.176859 832 raft_consensus.cc:2243] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.176992 832 raft_consensus.cc:2272] T 35dec88dbd394d4a80ee2e863aec6bbf P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.177381 832 tablet_replica.cc:333] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.177484 832 raft_consensus.cc:2243] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.177704 832 raft_consensus.cc:2272] T 881346b54fb94223a5246c0c38a20d91 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.178081 832 tablet_replica.cc:333] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.178197 832 raft_consensus.cc:2243] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.178354 832 raft_consensus.cc:2272] T 9f0b0e1a566f4bea899323654c7248a9 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.178720 832 tablet_replica.cc:333] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.178834 832 raft_consensus.cc:2243] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.178965 832 raft_consensus.cc:2272] T d0771ee252414935aabcc7a9aa6e922f P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.179349 832 tablet_replica.cc:333] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.179450 832 raft_consensus.cc:2243] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.179596 832 raft_consensus.cc:2272] T fc7584a5fb014c039b1a3f4197cd017a P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.179965 832 tablet_replica.cc:333] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.180066 832 raft_consensus.cc:2243] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.180207 832 raft_consensus.cc:2272] T bc238dfc34fc4acb9d646c80e38c01b9 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.180604 832 tablet_replica.cc:333] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.180717 832 raft_consensus.cc:2243] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.180927 832 raft_consensus.cc:2272] T 12a808ed739f4d159faa084914e966dd P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.181313 832 tablet_replica.cc:333] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.181447 832 raft_consensus.cc:2243] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.181595 832 raft_consensus.cc:2272] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.181979 832 tablet_replica.cc:333] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c: stopping tablet replica
I20251212 08:53:25.182107 832 raft_consensus.cc:2243] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.182330 832 raft_consensus.cc:2272] T 86f83858fbcd4ed18d56f089d34d7a98 P 7cab108f9abb4e1c973338419932d96c [term 1 FOLLOWER]: Raft consensus is shut down!
W20251212 08:53:25.187376 2843 consensus_peers.cc:597] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.187918 2843 consensus_peers.cc:597] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.193266 2843 consensus_peers.cc:597] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.201854 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
W20251212 08:53:25.211372 2843 consensus_peers.cc:597] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.225841 2763 consensus_peers.cc:597] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.225951 2763 consensus_peers.cc:597] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.228459 2763 consensus_peers.cc:597] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.230391 2843 consensus_peers.cc:597] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.238889 2843 consensus_peers.cc:597] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.244732 832 tablet_server.cc:178] TabletServer@127.0.208.2:0 shutting down...
W20251212 08:53:25.253369 2763 consensus_peers.cc:597] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.256026 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:25.256372 832 tablet_replica.cc:333] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.256549 832 raft_consensus.cc:2243] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.256719 832 raft_consensus.cc:2272] T 29970481f9134918855691767fe1af96 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.257301 832 tablet_replica.cc:333] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.257472 832 raft_consensus.cc:2243] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.257719 832 raft_consensus.cc:2272] T c537842e2a634782afb1ea70568d698c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.258227 832 tablet_replica.cc:333] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.258386 832 raft_consensus.cc:2243] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.258538 832 raft_consensus.cc:2272] T 63cc5ffd60084db6a67873d0f0b74c4a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.258947 832 tablet_replica.cc:333] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.259111 832 raft_consensus.cc:2243] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.259255 832 raft_consensus.cc:2272] T f2c37ac0543d4af8afa8efeb773dd11c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.259670 832 tablet_replica.cc:333] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.259802 832 raft_consensus.cc:2243] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.260020 832 raft_consensus.cc:2272] T e8f092c5f152452695390c6d8ec19515 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.260458 832 tablet_replica.cc:333] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.260617 832 raft_consensus.cc:2243] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.260798 832 raft_consensus.cc:2272] T c57c808afa054124984ee62dcaa4c22e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.261194 832 tablet_replica.cc:333] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.261333 832 raft_consensus.cc:2243] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.261477 832 raft_consensus.cc:2272] T c0aa78ae4efc4856a0262730d672874f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.261906 832 tablet_replica.cc:333] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.262059 832 raft_consensus.cc:2243] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.262346 832 raft_consensus.cc:2272] T bc958968094c4914ae8e620f1cfb94fa P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.262809 832 tablet_replica.cc:333] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.262984 832 raft_consensus.cc:2243] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.263154 832 raft_consensus.cc:2272] T 4802e69098d74c6eba89068ae8a73106 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251212 08:53:25.263429 2845 consensus_peers.cc:597] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 -> Peer 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879): Couldn't send request to peer 7aad0f0d166f4050b76cd4d94dbbbc78. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.263595 832 tablet_replica.cc:333] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.263732 832 raft_consensus.cc:2243] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.263948 832 raft_consensus.cc:2272] T e66ba33ab78f4604b175eef3351f38c1 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.264400 832 tablet_replica.cc:333] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.264540 832 raft_consensus.cc:2243] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.264680 832 raft_consensus.cc:2272] T d7e15336531948be810a2944afa986a4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.265081 832 tablet_replica.cc:333] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.265225 832 raft_consensus.cc:2243] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.265381 832 raft_consensus.cc:2272] T 9f3f58afa0ad41c6886c0306b7801ea6 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.265853 832 tablet_replica.cc:333] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.266006 832 raft_consensus.cc:2243] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.266163 832 raft_consensus.cc:2272] T de0691c0e98f4700b784a2981f741b5a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.266664 832 tablet_replica.cc:333] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.266829 832 raft_consensus.cc:2243] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.266996 832 raft_consensus.cc:2272] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.267454 832 tablet_replica.cc:333] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.267591 832 raft_consensus.cc:2243] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.267731 832 raft_consensus.cc:2272] T f03c86d8d28946a3b39af1b7d94a9226 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.268119 832 tablet_replica.cc:333] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.268246 832 raft_consensus.cc:2243] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.268508 832 raft_consensus.cc:2272] T e389399dd80a4284b150fc473d1928e8 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.268982 832 tablet_replica.cc:333] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.269125 832 raft_consensus.cc:2243] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.269351 832 raft_consensus.cc:2272] T db9329c939f846f48bd173a350bef36d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.269747 832 tablet_replica.cc:333] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.269888 832 raft_consensus.cc:2243] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.270031 832 raft_consensus.cc:2272] T c3c476911f1449d299b316d4d5730c71 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251212 08:53:25.270144 2763 consensus_peers.cc:597] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.270473 832 tablet_replica.cc:333] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.270607 832 raft_consensus.cc:2243] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.270740 832 raft_consensus.cc:2272] T 31333c1e9c574d0bb2f38ca25d972d70 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.271126 832 tablet_replica.cc:333] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.271270 832 raft_consensus.cc:2243] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.271418 832 raft_consensus.cc:2272] T 8459df9bf95e4b21abb8a96966cb0fb2 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.271878 832 tablet_replica.cc:333] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.272042 832 raft_consensus.cc:2243] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.272351 832 raft_consensus.cc:2272] T bcd3ea4b4e7341b59b92ce737d89af33 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.272863 832 tablet_replica.cc:333] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
W20251212 08:53:25.273021 2843 consensus_peers.cc:597] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.273036 832 raft_consensus.cc:2243] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.273247 832 raft_consensus.cc:2272] T 0f2534bb7ce049a19359b3c97a383cec P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.273674 832 tablet_replica.cc:333] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
W20251212 08:53:25.273779 2845 consensus_peers.cc:597] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 -> Peer 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879): Couldn't send request to peer 7aad0f0d166f4050b76cd4d94dbbbc78. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.273820 832 raft_consensus.cc:2243] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.273972 832 raft_consensus.cc:2272] T 38ebd93ba8814c59b510e570877bca91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.274395 832 tablet_replica.cc:333] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.274530 832 raft_consensus.cc:2243] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.274746 832 raft_consensus.cc:2272] T 337002230a3949a1af139ac4ac693f81 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.275141 832 tablet_replica.cc:333] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.275270 832 raft_consensus.cc:2243] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.275478 832 raft_consensus.cc:2272] T ff97ced42b1543b2a2b41d4106de4e5c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.275851 832 tablet_replica.cc:333] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.275985 832 raft_consensus.cc:2243] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.276129 832 raft_consensus.cc:2272] T 84832bfc9dd444a39aff9800980365ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.276563 832 tablet_replica.cc:333] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.276692 832 raft_consensus.cc:2243] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.276904 832 raft_consensus.cc:2272] T 2fd487ada3bd4c41b299172d81e63b7e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.277326 832 tablet_replica.cc:333] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.277441 832 raft_consensus.cc:2243] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.277581 832 raft_consensus.cc:2272] T 6b147d7c9a3c44808ebae11ec97725e0 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.277953 832 tablet_replica.cc:333] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.278079 832 raft_consensus.cc:2243] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.278231 832 raft_consensus.cc:2272] T 62b27193ad064f6e87f51a3e3a8756bc P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.278659 832 tablet_replica.cc:333] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.278785 832 raft_consensus.cc:2243] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.278918 832 raft_consensus.cc:2272] T d84415e201b34feb9991ac04c8355802 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.279306 832 tablet_replica.cc:333] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.279433 832 raft_consensus.cc:2243] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.279600 832 raft_consensus.cc:2272] T 82b6f18e15f941929f188b2b490718ca P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.280062 832 tablet_replica.cc:333] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.280201 832 raft_consensus.cc:2243] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.280486 832 raft_consensus.cc:2272] T 5a5ad45938984dbfbb2349fa3c85fccb P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.280987 832 tablet_replica.cc:333] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.281149 832 raft_consensus.cc:2243] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.281421 832 raft_consensus.cc:2272] T 92719eed5a0448f2a99ac8897be6c266 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.281920 832 tablet_replica.cc:333] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.282085 832 raft_consensus.cc:2243] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.282240 832 raft_consensus.cc:2272] T 888c406ad456495cb6bb53abac986a94 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.282756 832 tablet_replica.cc:333] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.282903 832 raft_consensus.cc:2243] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.283075 832 raft_consensus.cc:2272] T 50e34e7a14274ee9ad9d640eb15a6909 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.283561 832 tablet_replica.cc:333] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.283697 832 raft_consensus.cc:2243] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.283965 832 raft_consensus.cc:2272] T 7d16ad80c41b41e2accecd66ca20e04a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251212 08:53:25.284245 2845 consensus_peers.cc:597] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 -> Peer 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879): Couldn't send request to peer 7aad0f0d166f4050b76cd4d94dbbbc78. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.284476 832 tablet_replica.cc:333] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.284623 832 raft_consensus.cc:2243] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.284766 832 raft_consensus.cc:2272] T a9cc0c9708724db590f46d357c6a3639 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.285152 832 tablet_replica.cc:333] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.285295 832 raft_consensus.cc:2243] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.285429 832 raft_consensus.cc:2272] T dfc6271b43244577a6b05d092920f236 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.285835 832 tablet_replica.cc:333] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.285964 832 raft_consensus.cc:2243] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.286172 832 raft_consensus.cc:2272] T 47b760e1902f4f5f84cdb54634e6da97 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.286563 832 tablet_replica.cc:333] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.286712 832 raft_consensus.cc:2243] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.286849 832 raft_consensus.cc:2272] T c747caeb72d148f1a70b0ce32ea9f7ff P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.287235 832 tablet_replica.cc:333] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.287371 832 raft_consensus.cc:2243] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.287503 832 raft_consensus.cc:2272] T c40e9cbf2b184cb7be83ed003ecadfde P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.287865 832 tablet_replica.cc:333] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.287990 832 raft_consensus.cc:2243] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.288203 832 raft_consensus.cc:2272] T a295a2fa31fe405da6380ccfd082338b P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.288607 832 tablet_replica.cc:333] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.288738 832 raft_consensus.cc:2243] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.288935 832 raft_consensus.cc:2272] T 0e1c7c8118304721a5c37409bb220623 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.289327 832 tablet_replica.cc:333] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.289459 832 raft_consensus.cc:2243] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.289592 832 raft_consensus.cc:2272] T d897186bf4c3441caefa0c2a5d83fc4c P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.289956 832 tablet_replica.cc:333] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.290089 832 raft_consensus.cc:2243] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.290223 832 raft_consensus.cc:2272] T 8f66b4c1169b4621838e302888e31047 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.290599 832 tablet_replica.cc:333] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.290724 832 raft_consensus.cc:2243] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.290854 832 raft_consensus.cc:2272] T 0ebda846433246b0af04f3126d32f4ce P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.291224 832 tablet_replica.cc:333] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.291354 832 raft_consensus.cc:2243] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.291488 832 raft_consensus.cc:2272] T 9bbebd39db8945459705e59f8670eb3f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.291872 832 tablet_replica.cc:333] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.291996 832 raft_consensus.cc:2243] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.292200 832 raft_consensus.cc:2272] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.292610 832 tablet_replica.cc:333] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.292740 832 raft_consensus.cc:2243] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.292941 832 raft_consensus.cc:2272] T 85bd9f82cab946bfa260338f237a589d P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.293331 832 tablet_replica.cc:333] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.293454 832 raft_consensus.cc:2243] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.293689 832 raft_consensus.cc:2272] T 402f26e2e46f416cb4ee88c2f09a5dfd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.294148 832 tablet_replica.cc:333] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.294308 832 raft_consensus.cc:2243] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.294552 832 raft_consensus.cc:2272] T 8638463f714e4dddbf5cec9d89890e4e P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.295053 832 tablet_replica.cc:333] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.295200 832 raft_consensus.cc:2243] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.295490 832 raft_consensus.cc:2272] T 6d67e78322704917859cd6c6e62b6964 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.296048 832 tablet_replica.cc:333] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.296216 832 raft_consensus.cc:2243] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.296422 832 raft_consensus.cc:2272] T 881346b54fb94223a5246c0c38a20d91 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.296823 832 tablet_replica.cc:333] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.296948 832 raft_consensus.cc:2243] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.297184 832 raft_consensus.cc:2272] T 9f0b0e1a566f4bea899323654c7248a9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.297586 832 tablet_replica.cc:333] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.297732 832 raft_consensus.cc:2243] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.297884 832 raft_consensus.cc:2272] T c0c0068af3ec4fa1b34b33e05ec9da93 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.298326 832 tablet_replica.cc:333] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.298465 832 raft_consensus.cc:2243] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.298621 832 raft_consensus.cc:2272] T 35dec88dbd394d4a80ee2e863aec6bbf P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.299021 832 tablet_replica.cc:333] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.299161 832 raft_consensus.cc:2243] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.299324 832 raft_consensus.cc:2272] T d0771ee252414935aabcc7a9aa6e922f P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.299713 832 tablet_replica.cc:333] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.299851 832 raft_consensus.cc:2243] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.300004 832 raft_consensus.cc:2272] T fc7584a5fb014c039b1a3f4197cd017a P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.300472 832 tablet_replica.cc:333] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.300609 832 raft_consensus.cc:2243] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.300832 832 raft_consensus.cc:2272] T bc238dfc34fc4acb9d646c80e38c01b9 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.301232 832 tablet_replica.cc:333] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.301373 832 raft_consensus.cc:2243] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.301530 832 raft_consensus.cc:2272] T 12a808ed739f4d159faa084914e966dd P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.301923 832 tablet_replica.cc:333] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.302045 832 raft_consensus.cc:2243] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.302301 832 raft_consensus.cc:2272] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.302709 832 tablet_replica.cc:333] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.302824 832 raft_consensus.cc:2243] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.302992 832 raft_consensus.cc:2272] T 86f83858fbcd4ed18d56f089d34d7a98 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.303400 832 tablet_replica.cc:333] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.303525 832 raft_consensus.cc:2243] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.303753 832 raft_consensus.cc:2272] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.304152 832 tablet_replica.cc:333] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78: stopping tablet replica
I20251212 08:53:25.304255 832 raft_consensus.cc:2243] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.304474 832 raft_consensus.cc:2272] T d4841a0029994cc3a2d02e5889d34958 P 7aad0f0d166f4050b76cd4d94dbbbc78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.322165 832 tablet_server.cc:195] TabletServer@127.0.208.2:0 shutdown complete.
W20251212 08:53:25.334129 2843 consensus_peers.cc:597] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.343917 2843 consensus_peers.cc:597] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.344535 2843 consensus_peers.cc:597] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 -> Peer 7cab108f9abb4e1c973338419932d96c (127.0.208.1:35409): Couldn't send request to peer 7cab108f9abb4e1c973338419932d96c. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.1:35409: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251212 08:53:25.351876 2845 consensus_peers.cc:597] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 -> Peer 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879): Couldn't send request to peer 7aad0f0d166f4050b76cd4d94dbbbc78. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.2:41879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.361595 832 tablet_server.cc:178] TabletServer@127.0.208.3:0 shutting down...
I20251212 08:53:25.371290 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:25.371577 832 tablet_replica.cc:333] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.371735 832 raft_consensus.cc:2243] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.371927 832 raft_consensus.cc:2272] T 29970481f9134918855691767fe1af96 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.372501 832 tablet_replica.cc:333] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.372633 832 raft_consensus.cc:2243] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.372773 832 raft_consensus.cc:2272] T c537842e2a634782afb1ea70568d698c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.373165 832 tablet_replica.cc:333] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.373286 832 raft_consensus.cc:2243] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.373507 832 raft_consensus.cc:2272] T 63cc5ffd60084db6a67873d0f0b74c4a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.373867 832 tablet_replica.cc:333] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.373991 832 raft_consensus.cc:2243] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.399699 832 raft_consensus.cc:2272] T e8f092c5f152452695390c6d8ec19515 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.400435 832 tablet_replica.cc:333] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.400588 832 raft_consensus.cc:2243] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.400720 832 raft_consensus.cc:2272] T f2c37ac0543d4af8afa8efeb773dd11c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.401116 832 tablet_replica.cc:333] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.401235 832 raft_consensus.cc:2243] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.401351 832 raft_consensus.cc:2272] T c57c808afa054124984ee62dcaa4c22e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.401700 832 tablet_replica.cc:333] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.401811 832 raft_consensus.cc:2243] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.402056 832 raft_consensus.cc:2272] T c0aa78ae4efc4856a0262730d672874f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.402436 832 tablet_replica.cc:333] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.402549 832 raft_consensus.cc:2243] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.402665 832 raft_consensus.cc:2272] T bc958968094c4914ae8e620f1cfb94fa P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.403018 832 tablet_replica.cc:333] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.403136 832 raft_consensus.cc:2243] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.403249 832 raft_consensus.cc:2272] T 4802e69098d74c6eba89068ae8a73106 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.403594 832 tablet_replica.cc:333] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.403713 832 raft_consensus.cc:2243] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.403824 832 raft_consensus.cc:2272] T e66ba33ab78f4604b175eef3351f38c1 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.404174 832 tablet_replica.cc:333] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.404317 832 raft_consensus.cc:2243] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.404465 832 raft_consensus.cc:2272] T d7e15336531948be810a2944afa986a4 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.404805 832 tablet_replica.cc:333] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.404915 832 raft_consensus.cc:2243] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.405035 832 raft_consensus.cc:2272] T 9f3f58afa0ad41c6886c0306b7801ea6 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.405392 832 tablet_replica.cc:333] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.405503 832 raft_consensus.cc:2243] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.405612 832 raft_consensus.cc:2272] T de0691c0e98f4700b784a2981f741b5a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.405956 832 tablet_replica.cc:333] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.406065 832 raft_consensus.cc:2243] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.406289 832 raft_consensus.cc:2272] T bbe34e1fdbf24c1c9c1edc1e14bfa88d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.406641 832 tablet_replica.cc:333] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.406759 832 raft_consensus.cc:2243] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.406872 832 raft_consensus.cc:2272] T f03c86d8d28946a3b39af1b7d94a9226 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.407219 832 tablet_replica.cc:333] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.407321 832 raft_consensus.cc:2243] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.407430 832 raft_consensus.cc:2272] T e389399dd80a4284b150fc473d1928e8 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.407763 832 tablet_replica.cc:333] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.407863 832 raft_consensus.cc:2243] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.407980 832 raft_consensus.cc:2272] T db9329c939f846f48bd173a350bef36d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.408366 832 tablet_replica.cc:333] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.408478 832 raft_consensus.cc:2243] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.408591 832 raft_consensus.cc:2272] T c3c476911f1449d299b316d4d5730c71 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.408923 832 tablet_replica.cc:333] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.409085 832 raft_consensus.cc:2243] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.409298 832 raft_consensus.cc:2272] T 31333c1e9c574d0bb2f38ca25d972d70 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.409648 832 tablet_replica.cc:333] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.409762 832 raft_consensus.cc:2243] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.409874 832 raft_consensus.cc:2272] T 8459df9bf95e4b21abb8a96966cb0fb2 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.410216 832 tablet_replica.cc:333] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.410321 832 raft_consensus.cc:2243] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.410434 832 raft_consensus.cc:2272] T bcd3ea4b4e7341b59b92ce737d89af33 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.410846 832 tablet_replica.cc:333] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.411016 832 raft_consensus.cc:2243] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.411159 832 raft_consensus.cc:2272] T 0f2534bb7ce049a19359b3c97a383cec P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.411586 832 tablet_replica.cc:333] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.411759 832 raft_consensus.cc:2243] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.411943 832 raft_consensus.cc:2272] T 38ebd93ba8814c59b510e570877bca91 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251212 08:53:25.412168 2845 consensus_peers.cc:597] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 -> Peer 7aad0f0d166f4050b76cd4d94dbbbc78 (127.0.208.2:41879): Couldn't send request to peer 7aad0f0d166f4050b76cd4d94dbbbc78. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.2:41879: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:25.412431 832 tablet_replica.cc:333] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.412544 832 raft_consensus.cc:2243] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.412662 832 raft_consensus.cc:2272] T 337002230a3949a1af139ac4ac693f81 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.413055 832 tablet_replica.cc:333] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.413159 832 raft_consensus.cc:2243] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.413276 832 raft_consensus.cc:2272] T ff97ced42b1543b2a2b41d4106de4e5c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.413641 832 tablet_replica.cc:333] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.413757 832 raft_consensus.cc:2243] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.413967 832 raft_consensus.cc:2272] T 84832bfc9dd444a39aff9800980365ca P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.414335 832 tablet_replica.cc:333] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.414439 832 raft_consensus.cc:2243] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.414549 832 raft_consensus.cc:2272] T 2fd487ada3bd4c41b299172d81e63b7e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.414902 832 tablet_replica.cc:333] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.415019 832 raft_consensus.cc:2243] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.415207 832 raft_consensus.cc:2272] T 6b147d7c9a3c44808ebae11ec97725e0 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.415555 832 tablet_replica.cc:333] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.415656 832 raft_consensus.cc:2243] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.415830 832 raft_consensus.cc:2272] T 62b27193ad064f6e87f51a3e3a8756bc P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.416177 832 tablet_replica.cc:333] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.416303 832 raft_consensus.cc:2243] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.416440 832 raft_consensus.cc:2272] T d84415e201b34feb9991ac04c8355802 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.416776 832 tablet_replica.cc:333] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.416883 832 raft_consensus.cc:2243] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.417073 832 raft_consensus.cc:2272] T 82b6f18e15f941929f188b2b490718ca P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.417420 832 tablet_replica.cc:333] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.417527 832 raft_consensus.cc:2243] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.417639 832 raft_consensus.cc:2272] T 5a5ad45938984dbfbb2349fa3c85fccb P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.417984 832 tablet_replica.cc:333] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.418094 832 raft_consensus.cc:2243] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.418203 832 raft_consensus.cc:2272] T 92719eed5a0448f2a99ac8897be6c266 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.418542 832 tablet_replica.cc:333] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.418648 832 raft_consensus.cc:2243] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.418757 832 raft_consensus.cc:2272] T 888c406ad456495cb6bb53abac986a94 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.419100 832 tablet_replica.cc:333] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.419204 832 raft_consensus.cc:2243] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.419313 832 raft_consensus.cc:2272] T 47b760e1902f4f5f84cdb54634e6da97 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.419652 832 tablet_replica.cc:333] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.419760 832 raft_consensus.cc:2243] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.419869 832 raft_consensus.cc:2272] T c747caeb72d148f1a70b0ce32ea9f7ff P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.420222 832 tablet_replica.cc:333] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.420369 832 raft_consensus.cc:2243] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.420491 832 raft_consensus.cc:2272] T c40e9cbf2b184cb7be83ed003ecadfde P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.420823 832 tablet_replica.cc:333] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.420928 832 raft_consensus.cc:2243] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.421044 832 raft_consensus.cc:2272] T a295a2fa31fe405da6380ccfd082338b P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.421375 832 tablet_replica.cc:333] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.421481 832 raft_consensus.cc:2243] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.421588 832 raft_consensus.cc:2272] T 0e1c7c8118304721a5c37409bb220623 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.421931 832 tablet_replica.cc:333] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.422050 832 raft_consensus.cc:2243] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.422159 832 raft_consensus.cc:2272] T d897186bf4c3441caefa0c2a5d83fc4c P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.422544 832 tablet_replica.cc:333] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.422649 832 raft_consensus.cc:2243] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.422762 832 raft_consensus.cc:2272] T 50e34e7a14274ee9ad9d640eb15a6909 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.423113 832 tablet_replica.cc:333] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.423221 832 raft_consensus.cc:2243] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.423332 832 raft_consensus.cc:2272] T 7d16ad80c41b41e2accecd66ca20e04a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.423712 832 tablet_replica.cc:333] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.423822 832 raft_consensus.cc:2243] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.423931 832 raft_consensus.cc:2272] T a9cc0c9708724db590f46d357c6a3639 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.424322 832 tablet_replica.cc:333] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.424438 832 raft_consensus.cc:2243] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.424654 832 raft_consensus.cc:2272] T dfc6271b43244577a6b05d092920f236 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.425009 832 tablet_replica.cc:333] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.425115 832 raft_consensus.cc:2243] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.425302 832 raft_consensus.cc:2272] T 8f66b4c1169b4621838e302888e31047 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.425645 832 tablet_replica.cc:333] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.425745 832 raft_consensus.cc:2243] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.425922 832 raft_consensus.cc:2272] T 0ebda846433246b0af04f3126d32f4ce P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.426286 832 tablet_replica.cc:333] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.426398 832 raft_consensus.cc:2243] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.426614 832 raft_consensus.cc:2272] T 9bbebd39db8945459705e59f8670eb3f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.426997 832 tablet_replica.cc:333] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.427110 832 raft_consensus.cc:2243] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.427254 832 raft_consensus.cc:2272] T adaf3ccf3aa645bcb2064ce429ac8ee4 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.427632 832 tablet_replica.cc:333] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.427755 832 raft_consensus.cc:2243] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.427887 832 raft_consensus.cc:2272] T 85bd9f82cab946bfa260338f237a589d P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.428272 832 tablet_replica.cc:333] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.428453 832 raft_consensus.cc:2243] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.428588 832 raft_consensus.cc:2272] T 402f26e2e46f416cb4ee88c2f09a5dfd P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.428979 832 tablet_replica.cc:333] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.429101 832 raft_consensus.cc:2243] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.429235 832 raft_consensus.cc:2272] T 8638463f714e4dddbf5cec9d89890e4e P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.429605 832 tablet_replica.cc:333] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.429726 832 raft_consensus.cc:2243] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.429852 832 raft_consensus.cc:2272] T 6d67e78322704917859cd6c6e62b6964 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.430238 832 tablet_replica.cc:333] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.430359 832 raft_consensus.cc:2243] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.430493 832 raft_consensus.cc:2272] T 881346b54fb94223a5246c0c38a20d91 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.430863 832 tablet_replica.cc:333] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.430986 832 raft_consensus.cc:2243] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.431118 832 raft_consensus.cc:2272] T 9f0b0e1a566f4bea899323654c7248a9 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.431486 832 tablet_replica.cc:333] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.431608 832 raft_consensus.cc:2243] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.431849 832 raft_consensus.cc:2272] T c0c0068af3ec4fa1b34b33e05ec9da93 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.432232 832 tablet_replica.cc:333] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.432377 832 raft_consensus.cc:2243] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.432591 832 raft_consensus.cc:2272] T 35dec88dbd394d4a80ee2e863aec6bbf P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.432973 832 tablet_replica.cc:333] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.433089 832 raft_consensus.cc:2243] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.433281 832 raft_consensus.cc:2272] T d0771ee252414935aabcc7a9aa6e922f P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.433641 832 tablet_replica.cc:333] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.433756 832 raft_consensus.cc:2243] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.433959 832 raft_consensus.cc:2272] T fc7584a5fb014c039b1a3f4197cd017a P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.434362 832 tablet_replica.cc:333] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.434500 832 raft_consensus.cc:2243] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.434654 832 raft_consensus.cc:2272] T bc238dfc34fc4acb9d646c80e38c01b9 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.435051 832 tablet_replica.cc:333] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.435184 832 raft_consensus.cc:2243] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.435340 832 raft_consensus.cc:2272] T 701bc5e1a89b4e10bf2d468cf0c0e783 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.435750 832 tablet_replica.cc:333] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.435892 832 raft_consensus.cc:2243] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.436059 832 raft_consensus.cc:2272] T 86f83858fbcd4ed18d56f089d34d7a98 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.436504 832 tablet_replica.cc:333] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.436640 832 raft_consensus.cc:2243] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.436793 832 raft_consensus.cc:2272] T 12a808ed739f4d159faa084914e966dd P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.437187 832 tablet_replica.cc:333] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.437317 832 raft_consensus.cc:2243] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:25.437464 832 raft_consensus.cc:2272] T ab43cb77736b4eaf9dd5c6bd73cfc733 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.437856 832 tablet_replica.cc:333] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132: stopping tablet replica
I20251212 08:53:25.437989 832 raft_consensus.cc:2243] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.438216 832 raft_consensus.cc:2272] T d4841a0029994cc3a2d02e5889d34958 P 2d41c990aa9f453e96ec315fab508132 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.465670 832 tablet_server.cc:195] TabletServer@127.0.208.3:0 shutdown complete.
I20251212 08:53:25.501600 832 master.cc:561] Master@127.0.208.62:43207 shutting down...
I20251212 08:53:25.512476 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.512753 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.512846 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P a3f2c4b47cab4b1bb6e0db7a989eb698: stopping tablet replica
I20251212 08:53:25.536568 832 master.cc:583] Master@127.0.208.62:43207 shutdown complete.
[ OK ] ClientTest.TestAutoFlushBackgroundAndErrorCollector (43098 ms)
[ RUN ] ClientTest.TestMutationsWork
I20251212 08:53:25.594647 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:44263
I20251212 08:53:25.595178 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:25.597829 3978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:25.598057 832 server_base.cc:1047] running on GCE node
W20251212 08:53:25.599339 3976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:25.599341 3975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:25.599776 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:25.599846 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:25.599884 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529605599868 us; error 0 us; skew 500 ppm
I20251212 08:53:25.600044 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:25.602342 832 webserver.cc:492] Webserver started at http://127.0.208.62:33463/ using document root <none> and password file <none>
I20251212 08:53:25.602552 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:25.602629 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:25.602763 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:25.603408 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "f456e50a31c24c4ebba6d4a58803bc91"
format_stamp: "Formatted at 2025-12-12 08:53:25 on dist-test-slave-qdpb"
I20251212 08:53:25.605860 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:53:25.607394 3983 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:25.607702 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:53:25.629029 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "f456e50a31c24c4ebba6d4a58803bc91"
format_stamp: "Formatted at 2025-12-12 08:53:25 on dist-test-slave-qdpb"
I20251212 08:53:25.629211 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:25.637768 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:25.638283 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:25.650933 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:44263
I20251212 08:53:25.650947 4044 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:44263 every 8 connection(s)
I20251212 08:53:25.652228 4045 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:25.656515 4045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91: Bootstrap starting.
I20251212 08:53:25.657819 4045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:25.659375 4045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91: No bootstrap required, opened a new log
I20251212 08:53:25.659936 4045 raft_consensus.cc:359] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f456e50a31c24c4ebba6d4a58803bc91" member_type: VOTER }
I20251212 08:53:25.660070 4045 raft_consensus.cc:385] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:25.660111 4045 raft_consensus.cc:740] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f456e50a31c24c4ebba6d4a58803bc91, State: Initialized, Role: FOLLOWER
I20251212 08:53:25.660310 4045 consensus_queue.cc:260] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f456e50a31c24c4ebba6d4a58803bc91" member_type: VOTER }
I20251212 08:53:25.660427 4045 raft_consensus.cc:399] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:25.660463 4045 raft_consensus.cc:493] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:25.660516 4045 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:25.662326 4045 raft_consensus.cc:515] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f456e50a31c24c4ebba6d4a58803bc91" member_type: VOTER }
I20251212 08:53:25.662492 4045 leader_election.cc:304] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f456e50a31c24c4ebba6d4a58803bc91; no voters:
I20251212 08:53:25.662837 4045 leader_election.cc:290] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:25.663012 4048 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:25.663424 4048 raft_consensus.cc:697] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 1 LEADER]: Becoming Leader. State: Replica: f456e50a31c24c4ebba6d4a58803bc91, State: Running, Role: LEADER
I20251212 08:53:25.663636 4048 consensus_queue.cc:237] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f456e50a31c24c4ebba6d4a58803bc91" member_type: VOTER }
I20251212 08:53:25.663717 4045 sys_catalog.cc:565] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:25.664398 4050 sys_catalog.cc:455] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f456e50a31c24c4ebba6d4a58803bc91. Latest consensus state: current_term: 1 leader_uuid: "f456e50a31c24c4ebba6d4a58803bc91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f456e50a31c24c4ebba6d4a58803bc91" member_type: VOTER } }
I20251212 08:53:25.664541 4050 sys_catalog.cc:458] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:25.664763 4049 sys_catalog.cc:455] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f456e50a31c24c4ebba6d4a58803bc91" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f456e50a31c24c4ebba6d4a58803bc91" member_type: VOTER } }
I20251212 08:53:25.664987 4049 sys_catalog.cc:458] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:25.666237 4053 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:25.668013 4053 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:25.668777 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:25.670951 4053 catalog_manager.cc:1357] Generated new cluster ID: 31a77976d1cd4999a363028e3ae7bb88
I20251212 08:53:25.671021 4053 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:25.689237 4053 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:25.690043 4053 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:25.703924 4053 catalog_manager.cc:6027] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91: Generated new TSK 0
I20251212 08:53:25.704241 4053 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:25.733848 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:25.736529 4066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:25.736608 4067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:25.737113 4069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:25.737373 832 server_base.cc:1047] running on GCE node
I20251212 08:53:25.737602 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:25.737654 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:25.737676 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529605737673 us; error 0 us; skew 500 ppm
I20251212 08:53:25.737780 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:25.738991 832 webserver.cc:492] Webserver started at http://127.0.208.1:46041/ using document root <none> and password file <none>
I20251212 08:53:25.739174 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:25.739235 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:25.739327 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:25.739905 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "9728f417639841448d83d6372a51b22b"
format_stamp: "Formatted at 2025-12-12 08:53:25 on dist-test-slave-qdpb"
I20251212 08:53:25.742344 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:25.743853 4074 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:25.744184 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:25.744305 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "9728f417639841448d83d6372a51b22b"
format_stamp: "Formatted at 2025-12-12 08:53:25 on dist-test-slave-qdpb"
I20251212 08:53:25.744413 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestMutationsWork.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:25.775758 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:25.776355 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:25.778116 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:25.778183 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:25.778272 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:25.778321 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:25.800555 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:40849
I20251212 08:53:25.800591 4144 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:40849 every 8 connection(s)
I20251212 08:53:25.802058 4145 heartbeater.cc:344] Connected to a master server at 127.0.208.62:44263
I20251212 08:53:25.802183 4145 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:25.802480 4145 heartbeater.cc:507] Master 127.0.208.62:44263 requested a full tablet report, sending...
I20251212 08:53:25.803234 4000 ts_manager.cc:194] Registered new tserver with Master: 9728f417639841448d83d6372a51b22b (127.0.208.1:40849)
I20251212 08:53:25.803742 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002339868s
I20251212 08:53:25.804313 4000 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:41100
I20251212 08:53:25.814291 4000 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41118:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:25.815605 4145 heartbeater.cc:499] Master 127.0.208.62:44263 was elected leader, sending a full tablet report...
I20251212 08:53:25.827466 4105 tablet_service.cc:1505] Processing CreateTablet for tablet a806c9f659d845f798530aeded904cab (DEFAULT_TABLE table=client-testtb [id=3b06e49ee075495da0509b4c3522bc1f]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:25.827484 4104 tablet_service.cc:1505] Processing CreateTablet for tablet 3d2600849f394f85bd0e8c0e379649cd (DEFAULT_TABLE table=client-testtb [id=3b06e49ee075495da0509b4c3522bc1f]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:25.827931 4105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a806c9f659d845f798530aeded904cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:25.828171 4104 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3d2600849f394f85bd0e8c0e379649cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:25.833212 4155 tablet_bootstrap.cc:492] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b: Bootstrap starting.
I20251212 08:53:25.834694 4155 tablet_bootstrap.cc:654] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:25.836447 4155 tablet_bootstrap.cc:492] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b: No bootstrap required, opened a new log
I20251212 08:53:25.836571 4155 ts_tablet_manager.cc:1403] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251212 08:53:25.837144 4155 raft_consensus.cc:359] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } }
I20251212 08:53:25.837285 4155 raft_consensus.cc:385] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:25.837345 4155 raft_consensus.cc:740] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9728f417639841448d83d6372a51b22b, State: Initialized, Role: FOLLOWER
I20251212 08:53:25.837522 4155 consensus_queue.cc:260] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } }
I20251212 08:53:25.837633 4155 raft_consensus.cc:399] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:25.837692 4155 raft_consensus.cc:493] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:25.837766 4155 raft_consensus.cc:3060] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:25.839596 4155 raft_consensus.cc:515] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } }
I20251212 08:53:25.839787 4155 leader_election.cc:304] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9728f417639841448d83d6372a51b22b; no voters:
I20251212 08:53:25.840251 4155 leader_election.cc:290] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:25.840394 4157 raft_consensus.cc:2804] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:25.841135 4155 ts_tablet_manager.cc:1434] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20251212 08:53:25.841522 4155 tablet_bootstrap.cc:492] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b: Bootstrap starting.
I20251212 08:53:25.844422 4155 tablet_bootstrap.cc:654] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:25.844806 4157 raft_consensus.cc:697] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 1 LEADER]: Becoming Leader. State: Replica: 9728f417639841448d83d6372a51b22b, State: Running, Role: LEADER
I20251212 08:53:25.845221 4157 consensus_queue.cc:237] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } }
I20251212 08:53:25.847311 4155 tablet_bootstrap.cc:492] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b: No bootstrap required, opened a new log
I20251212 08:53:25.847466 4155 ts_tablet_manager.cc:1403] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20251212 08:53:25.847676 3999 catalog_manager.cc:5654] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9728f417639841448d83d6372a51b22b (127.0.208.1). New cstate: current_term: 1 leader_uuid: "9728f417639841448d83d6372a51b22b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:25.848229 4155 raft_consensus.cc:359] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } }
I20251212 08:53:25.848409 4155 raft_consensus.cc:385] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:25.848472 4155 raft_consensus.cc:740] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9728f417639841448d83d6372a51b22b, State: Initialized, Role: FOLLOWER
I20251212 08:53:25.848654 4155 consensus_queue.cc:260] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } }
I20251212 08:53:25.848771 4155 raft_consensus.cc:399] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:25.848836 4155 raft_consensus.cc:493] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:25.848913 4155 raft_consensus.cc:3060] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:25.851120 4155 raft_consensus.cc:515] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } }
I20251212 08:53:25.851308 4155 leader_election.cc:304] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9728f417639841448d83d6372a51b22b; no voters:
I20251212 08:53:25.851451 4155 leader_election.cc:290] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:25.851512 4157 raft_consensus.cc:2804] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:25.851671 4157 raft_consensus.cc:697] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 1 LEADER]: Becoming Leader. State: Replica: 9728f417639841448d83d6372a51b22b, State: Running, Role: LEADER
I20251212 08:53:25.851852 4155 ts_tablet_manager.cc:1434] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:25.851928 4157 consensus_queue.cc:237] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } }
I20251212 08:53:25.854116 3999 catalog_manager.cc:5654] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b reported cstate change: term changed from 0 to 1, leader changed from <none> to 9728f417639841448d83d6372a51b22b (127.0.208.1). New cstate: current_term: 1 leader_uuid: "9728f417639841448d83d6372a51b22b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9728f417639841448d83d6372a51b22b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40849 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:25.896714 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:25.905843 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:25.906208 832 tablet_replica.cc:333] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b: stopping tablet replica
I20251212 08:53:25.906401 832 raft_consensus.cc:2243] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.906567 832 raft_consensus.cc:2272] T 3d2600849f394f85bd0e8c0e379649cd P 9728f417639841448d83d6372a51b22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.907377 832 tablet_replica.cc:333] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b: stopping tablet replica
I20251212 08:53:25.907487 832 raft_consensus.cc:2243] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.907644 832 raft_consensus.cc:2272] T a806c9f659d845f798530aeded904cab P 9728f417639841448d83d6372a51b22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.922639 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:25.927135 832 master.cc:561] Master@127.0.208.62:44263 shutting down...
I20251212 08:53:25.936923 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:25.937201 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:25.937404 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P f456e50a31c24c4ebba6d4a58803bc91: stopping tablet replica
I20251212 08:53:25.951743 832 master.cc:583] Master@127.0.208.62:44263 shutdown complete.
[ OK ] ClientTest.TestMutationsWork (392 ms)
[ RUN ] ClientTest.TestSoftDeleteAndReserveTable
I20251212 08:53:25.986820 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:33639
I20251212 08:53:25.987340 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:25.989795 4166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:25.989854 4167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:25.990271 4169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:25.990443 832 server_base.cc:1047] running on GCE node
I20251212 08:53:25.990662 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:25.990718 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:25.990759 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529605990755 us; error 0 us; skew 500 ppm
I20251212 08:53:25.990909 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:25.992105 832 webserver.cc:492] Webserver started at http://127.0.208.62:36263/ using document root <none> and password file <none>
I20251212 08:53:25.992324 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:25.992388 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:25.992506 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:25.993124 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "58f01032886143b5801e558da9a6daf1"
format_stamp: "Formatted at 2025-12-12 08:53:25 on dist-test-slave-qdpb"
I20251212 08:53:25.995561 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:25.997298 4174 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:25.997750 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:25.997870 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "58f01032886143b5801e558da9a6daf1"
format_stamp: "Formatted at 2025-12-12 08:53:25 on dist-test-slave-qdpb"
I20251212 08:53:25.997987 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:26.006332 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:26.006872 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:26.021791 4235 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:33639 every 8 connection(s)
I20251212 08:53:26.021767 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:33639
I20251212 08:53:26.023126 4236 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:26.027730 4236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1: Bootstrap starting.
I20251212 08:53:26.029119 4236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:26.030812 4236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1: No bootstrap required, opened a new log
I20251212 08:53:26.031450 4236 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f01032886143b5801e558da9a6daf1" member_type: VOTER }
I20251212 08:53:26.031576 4236 raft_consensus.cc:385] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:26.031610 4236 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58f01032886143b5801e558da9a6daf1, State: Initialized, Role: FOLLOWER
I20251212 08:53:26.031805 4236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f01032886143b5801e558da9a6daf1" member_type: VOTER }
I20251212 08:53:26.031908 4236 raft_consensus.cc:399] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:26.031942 4236 raft_consensus.cc:493] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:26.032007 4236 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:26.034329 4236 raft_consensus.cc:515] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f01032886143b5801e558da9a6daf1" member_type: VOTER }
I20251212 08:53:26.034533 4236 leader_election.cc:304] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58f01032886143b5801e558da9a6daf1; no voters:
I20251212 08:53:26.035048 4236 leader_election.cc:290] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:26.035189 4239 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:26.035746 4239 raft_consensus.cc:697] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 1 LEADER]: Becoming Leader. State: Replica: 58f01032886143b5801e558da9a6daf1, State: Running, Role: LEADER
I20251212 08:53:26.035825 4236 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:26.036015 4239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f01032886143b5801e558da9a6daf1" member_type: VOTER }
I20251212 08:53:26.037005 4241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58f01032886143b5801e558da9a6daf1. Latest consensus state: current_term: 1 leader_uuid: "58f01032886143b5801e558da9a6daf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f01032886143b5801e558da9a6daf1" member_type: VOTER } }
I20251212 08:53:26.037207 4241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:26.036998 4240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58f01032886143b5801e558da9a6daf1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58f01032886143b5801e558da9a6daf1" member_type: VOTER } }
I20251212 08:53:26.037492 4240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:26.038221 4244 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:26.040232 4244 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:26.041051 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:26.044883 4244 catalog_manager.cc:1357] Generated new cluster ID: 11f97899bd4d4cd29aa0eeef2b1e942d
I20251212 08:53:26.045006 4244 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:26.079977 4244 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:26.080910 4244 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:26.088876 4244 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1: Generated new TSK 0
I20251212 08:53:26.089128 4244 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:26.107146 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:26.109922 4257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:26.110029 4258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:26.110685 832 server_base.cc:1047] running on GCE node
W20251212 08:53:26.110704 4260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:26.111083 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:26.111191 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:26.111261 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529606111240 us; error 0 us; skew 500 ppm
I20251212 08:53:26.111430 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:26.112643 832 webserver.cc:492] Webserver started at http://127.0.208.1:33185/ using document root <none> and password file <none>
I20251212 08:53:26.112847 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:26.112933 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:26.113065 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:26.113696 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "456555b856324176be09b64c4380105a"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.116080 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:26.117652 4265 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.117966 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:26.118078 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "456555b856324176be09b64c4380105a"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.118201 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:26.134675 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:26.135232 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:26.136868 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:26.136991 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.137164 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:26.137253 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.156708 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:34481
I20251212 08:53:26.156733 4335 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:34481 every 8 connection(s)
I20251212 08:53:26.158455 4336 heartbeater.cc:344] Connected to a master server at 127.0.208.62:33639
I20251212 08:53:26.158592 4336 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:26.158929 4336 heartbeater.cc:507] Master 127.0.208.62:33639 requested a full tablet report, sending...
I20251212 08:53:26.159740 4191 ts_manager.cc:194] Registered new tserver with Master: 456555b856324176be09b64c4380105a (127.0.208.1:34481)
I20251212 08:53:26.159965 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002371234s
I20251212 08:53:26.161317 4191 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43638
I20251212 08:53:26.170751 4191 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43648:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:26.172715 4336 heartbeater.cc:499] Master 127.0.208.62:33639 was elected leader, sending a full tablet report...
I20251212 08:53:26.184393 4298 tablet_service.cc:1505] Processing CreateTablet for tablet dbf096375d234885bfab55a34ef73114 (DEFAULT_TABLE table=client-testtb [id=b4c70d90820e4d7fb89c2e58c9e55d65]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:26.184489 4296 tablet_service.cc:1505] Processing CreateTablet for tablet 31596225c54a4af2a214353b039d0ab0 (DEFAULT_TABLE table=client-testtb [id=b4c70d90820e4d7fb89c2e58c9e55d65]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:26.184930 4298 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dbf096375d234885bfab55a34ef73114. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:26.185199 4296 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31596225c54a4af2a214353b039d0ab0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:26.190407 4346 tablet_bootstrap.cc:492] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: Bootstrap starting.
I20251212 08:53:26.191823 4346 tablet_bootstrap.cc:654] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:26.193679 4346 tablet_bootstrap.cc:492] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: No bootstrap required, opened a new log
I20251212 08:53:26.193806 4346 ts_tablet_manager.cc:1403] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251212 08:53:26.194381 4346 raft_consensus.cc:359] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } }
I20251212 08:53:26.194519 4346 raft_consensus.cc:385] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:26.194578 4346 raft_consensus.cc:740] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456555b856324176be09b64c4380105a, State: Initialized, Role: FOLLOWER
I20251212 08:53:26.194737 4346 consensus_queue.cc:260] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } }
I20251212 08:53:26.194844 4346 raft_consensus.cc:399] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:26.194896 4346 raft_consensus.cc:493] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:26.194962 4346 raft_consensus.cc:3060] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:26.196888 4346 raft_consensus.cc:515] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } }
I20251212 08:53:26.197126 4346 leader_election.cc:304] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 456555b856324176be09b64c4380105a; no voters:
I20251212 08:53:26.197840 4346 leader_election.cc:290] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:26.198000 4348 raft_consensus.cc:2804] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:26.198462 4348 raft_consensus.cc:697] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 1 LEADER]: Becoming Leader. State: Replica: 456555b856324176be09b64c4380105a, State: Running, Role: LEADER
I20251212 08:53:26.198622 4346 ts_tablet_manager.cc:1434] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:53:26.198768 4348 consensus_queue.cc:237] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } }
I20251212 08:53:26.198910 4346 tablet_bootstrap.cc:492] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: Bootstrap starting.
I20251212 08:53:26.200600 4346 tablet_bootstrap.cc:654] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:26.201172 4191 catalog_manager.cc:5654] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a reported cstate change: term changed from 0 to 1, leader changed from <none> to 456555b856324176be09b64c4380105a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "456555b856324176be09b64c4380105a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:26.203013 4346 tablet_bootstrap.cc:492] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: No bootstrap required, opened a new log
I20251212 08:53:26.203209 4346 ts_tablet_manager.cc:1403] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:26.203984 4346 raft_consensus.cc:359] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } }
I20251212 08:53:26.204211 4346 raft_consensus.cc:385] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:26.204335 4346 raft_consensus.cc:740] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456555b856324176be09b64c4380105a, State: Initialized, Role: FOLLOWER
I20251212 08:53:26.204632 4346 consensus_queue.cc:260] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } }
I20251212 08:53:26.204833 4346 raft_consensus.cc:399] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:26.204950 4346 raft_consensus.cc:493] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:26.205080 4346 raft_consensus.cc:3060] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:26.208191 4346 raft_consensus.cc:515] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } }
I20251212 08:53:26.208425 4346 leader_election.cc:304] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 456555b856324176be09b64c4380105a; no voters:
I20251212 08:53:26.208613 4346 leader_election.cc:290] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:26.208673 4348 raft_consensus.cc:2804] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:26.208824 4348 raft_consensus.cc:697] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 1 LEADER]: Becoming Leader. State: Replica: 456555b856324176be09b64c4380105a, State: Running, Role: LEADER
I20251212 08:53:26.209084 4346 ts_tablet_manager.cc:1434] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20251212 08:53:26.209055 4348 consensus_queue.cc:237] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } }
I20251212 08:53:26.211076 4191 catalog_manager.cc:5654] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a reported cstate change: term changed from 0 to 1, leader changed from <none> to 456555b856324176be09b64c4380105a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "456555b856324176be09b64c4380105a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456555b856324176be09b64c4380105a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 34481 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:26.243764 4188 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43648:
table { table_name: "client-testtb" } modify_external_catalogs: true reserve_seconds: 600
I20251212 08:53:26.247375 4188 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43648:
table { table_name: "client-testtb" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20251212 08:53:26.252851 4358 tablet.cc:1722] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251212 08:53:26.252986 4357 tablet.cc:1722] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 0 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 1
I20251212 08:53:26.255717 4188 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43648:
table { table_name: "client-testtb" } new_table_name: "client-testtb" modify_external_catalogs: true
I20251212 08:53:26.257912 4188 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43648:
table { table_name: "client-testtb" } alter_schema_steps { type: DROP_COLUMN drop_column { name: "int_val" } } alter_schema_steps { type: ADD_COLUMN add_column { schema { name: "new_col" type: INT32 is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 immutable: false } } } modify_external_catalogs: true
I20251212 08:53:26.259178 4188 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43648:
table { table_name: "client-testtb" } modify_external_catalogs: true reserve_seconds: 600
I20251212 08:53:26.260646 4188 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43648:
table { table_name: "client-testtb" } modify_external_catalogs: true new_extra_configs { key: "kudu.table.maintenance_priority" value: "3" }
I20251212 08:53:26.261651 4191 catalog_manager.cc:6199] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1: client-testtb [id=b4c70d90820e4d7fb89c2e58c9e55d65] alter complete (version 1)
I20251212 08:53:26.273937 4188 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43648:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:26.276186 4188 catalog_manager.cc:2894] Servicing RecallDeletedTableRpc request from {username='slave'} at 127.0.0.1:43648:
table { table_id: "b4c70d90820e4d7fb89c2e58c9e55d65" }
I20251212 08:53:26.279969 4188 catalog_manager.cc:3527] Servicing AlterTable request from {username='slave'} at 127.0.0.1:43648:
table { table_id: "b4c70d90820e4d7fb89c2e58c9e55d65" } new_extra_configs { key: "kudu.table.disable_compaction" value: "false" }
I20251212 08:53:26.285058 4357 tablet.cc:1722] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251212 08:53:26.285156 4358 tablet.cc:1722] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: Alter schema from (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 1 to (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
) version 2
I20251212 08:53:26.286485 4188 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:43648:
table { table_name: "client-testtb" } modify_external_catalogs: true reserve_seconds: 0
I20251212 08:53:26.286684 4188 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:43648:
table { table_name: "client-testtb" } modify_external_catalogs: true reserve_seconds: 0
I20251212 08:53:26.290901 4188 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1: Sending DeleteTablet for 1 replicas of tablet dbf096375d234885bfab55a34ef73114
I20251212 08:53:26.291200 4188 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1: Sending DeleteTablet for 1 replicas of tablet 31596225c54a4af2a214353b039d0ab0
I20251212 08:53:26.291356 4298 tablet_service.cc:1552] Processing DeleteTablet for tablet dbf096375d234885bfab55a34ef73114 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:53:26 UTC) from {username='slave'} at 127.0.0.1:57684
I20251212 08:53:26.291596 4296 tablet_service.cc:1552] Processing DeleteTablet for tablet dbf096375d234885bfab55a34ef73114 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:53:26 UTC) from {username='slave'} at 127.0.0.1:57684
I20251212 08:53:26.292331 4296 tablet_service.cc:1552] Processing DeleteTablet for tablet 31596225c54a4af2a214353b039d0ab0 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:53:26 UTC) from {username='slave'} at 127.0.0.1:57684
I20251212 08:53:26.292691 4359 tablet_replica.cc:333] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: stopping tablet replica
I20251212 08:53:26.292896 4359 raft_consensus.cc:2243] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:26.293190 4359 raft_consensus.cc:2272] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:26.294108 4359 ts_tablet_manager.cc:1916] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:53:26.295586 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:26.300576 4359 ts_tablet_manager.cc:1929] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.4
I20251212 08:53:26.300730 4359 log.cc:1199] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/ts-0-root/wals/dbf096375d234885bfab55a34ef73114
I20251212 08:53:26.301115 4359 ts_tablet_manager.cc:1950] T dbf096375d234885bfab55a34ef73114 P 456555b856324176be09b64c4380105a: Deleting consensus metadata
I20251212 08:53:26.302695 4359 tablet_replica.cc:333] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: stopping tablet replica
I20251212 08:53:26.302932 4359 raft_consensus.cc:2243] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:26.302976 4176 catalog_manager.cc:4985] TS 456555b856324176be09b64c4380105a (127.0.208.1:34481): tablet dbf096375d234885bfab55a34ef73114 (table client-testtb [id=b4c70d90820e4d7fb89c2e58c9e55d65]) successfully deleted
I20251212 08:53:26.303189 4359 raft_consensus.cc:2272] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a [term 1 FOLLOWER]: Raft consensus is shut down!
W20251212 08:53:26.303294 4176 catalog_manager.cc:4952] TS 456555b856324176be09b64c4380105a (127.0.208.1:34481): delete failed for tablet dbf096375d234885bfab55a34ef73114 because the tablet was not found. No further retry: Not found: Tablet not found: dbf096375d234885bfab55a34ef73114
I20251212 08:53:26.304481 4359 ts_tablet_manager.cc:1916] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:53:26.307476 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:26.310230 4359 ts_tablet_manager.cc:1929] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20251212 08:53:26.310343 4359 log.cc:1199] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestSoftDeleteAndReserveTable.1765529557162900-832-0/minicluster-data/ts-0-root/wals/31596225c54a4af2a214353b039d0ab0
I20251212 08:53:26.310653 4359 ts_tablet_manager.cc:1950] T 31596225c54a4af2a214353b039d0ab0 P 456555b856324176be09b64c4380105a: Deleting consensus metadata
W20251212 08:53:26.311338 4176 catalog_manager.cc:4977] TS 456555b856324176be09b64c4380105a (127.0.208.1:34481): delete failed for tablet 31596225c54a4af2a214353b039d0ab0 with error code TABLET_NOT_RUNNING: Service unavailable: Tablet Manager is not running: MANAGER_QUIESCING
I20251212 08:53:26.325393 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:26.349486 832 master.cc:561] Master@127.0.208.62:33639 shutting down...
W20251212 08:53:26.359445 4176 catalog_manager.cc:4712] TS 456555b856324176be09b64c4380105a (127.0.208.1:34481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 31596225c54a4af2a214353b039d0ab0: Network error: Client connection negotiation failed: client connection to 127.0.208.1:34481: connect: Connection refused (error 111)
I20251212 08:53:26.442010 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:26.442371 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:26.442580 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 58f01032886143b5801e558da9a6daf1: stopping tablet replica
I20251212 08:53:26.457769 832 master.cc:583] Master@127.0.208.62:33639 shutdown complete.
[ OK ] ClientTest.TestSoftDeleteAndReserveTable (480 ms)
[ RUN ] ClientTest.TestGetTableSchema
I20251212 08:53:26.467518 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:39423
I20251212 08:53:26.468192 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:26.470485 4361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:26.470480 4362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:26.471026 4364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:26.471215 832 server_base.cc:1047] running on GCE node
I20251212 08:53:26.471451 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:26.471503 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:26.471532 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529606471529 us; error 0 us; skew 500 ppm
I20251212 08:53:26.471652 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:26.472879 832 webserver.cc:492] Webserver started at http://127.0.208.62:39735/ using document root <none> and password file <none>
I20251212 08:53:26.473075 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:26.473146 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:26.473253 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:26.473814 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "52271d97bd014e7ca1a5ae9efa243fdc"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.476204 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:26.477937 4369 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.478307 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:26.478410 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "52271d97bd014e7ca1a5ae9efa243fdc"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.478518 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:26.487612 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:26.488196 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:26.502485 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:39423
I20251212 08:53:26.502507 4430 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:39423 every 8 connection(s)
I20251212 08:53:26.503774 4431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:26.508071 4431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc: Bootstrap starting.
I20251212 08:53:26.509406 4431 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:26.510990 4431 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc: No bootstrap required, opened a new log
I20251212 08:53:26.511765 4431 raft_consensus.cc:359] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52271d97bd014e7ca1a5ae9efa243fdc" member_type: VOTER }
I20251212 08:53:26.511900 4431 raft_consensus.cc:385] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:26.511938 4431 raft_consensus.cc:740] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52271d97bd014e7ca1a5ae9efa243fdc, State: Initialized, Role: FOLLOWER
I20251212 08:53:26.512280 4431 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52271d97bd014e7ca1a5ae9efa243fdc" member_type: VOTER }
I20251212 08:53:26.512449 4431 raft_consensus.cc:399] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:26.512491 4431 raft_consensus.cc:493] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:26.512552 4431 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:26.514529 4431 raft_consensus.cc:515] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52271d97bd014e7ca1a5ae9efa243fdc" member_type: VOTER }
I20251212 08:53:26.514715 4431 leader_election.cc:304] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52271d97bd014e7ca1a5ae9efa243fdc; no voters:
I20251212 08:53:26.515089 4431 leader_election.cc:290] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:26.515314 4434 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:26.515789 4434 raft_consensus.cc:697] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 1 LEADER]: Becoming Leader. State: Replica: 52271d97bd014e7ca1a5ae9efa243fdc, State: Running, Role: LEADER
I20251212 08:53:26.515957 4431 sys_catalog.cc:565] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:26.516034 4434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52271d97bd014e7ca1a5ae9efa243fdc" member_type: VOTER }
I20251212 08:53:26.517158 4436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52271d97bd014e7ca1a5ae9efa243fdc. Latest consensus state: current_term: 1 leader_uuid: "52271d97bd014e7ca1a5ae9efa243fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52271d97bd014e7ca1a5ae9efa243fdc" member_type: VOTER } }
I20251212 08:53:26.517338 4436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:26.517185 4435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52271d97bd014e7ca1a5ae9efa243fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52271d97bd014e7ca1a5ae9efa243fdc" member_type: VOTER } }
I20251212 08:53:26.517514 4435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:26.518350 4439 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:26.520536 4439 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:26.523885 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:26.524116 4439 catalog_manager.cc:1357] Generated new cluster ID: 41658952458344739d642ffa7cbada22
I20251212 08:53:26.524175 4439 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:26.555940 4439 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:26.556792 4439 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:26.564358 4439 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc: Generated new TSK 0
I20251212 08:53:26.564606 4439 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:26.588878 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:26.591580 4452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:26.591650 4453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:26.592572 4455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:26.592831 832 server_base.cc:1047] running on GCE node
I20251212 08:53:26.593055 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:26.593106 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:26.593130 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529606593126 us; error 0 us; skew 500 ppm
I20251212 08:53:26.593277 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:26.594522 832 webserver.cc:492] Webserver started at http://127.0.208.1:46269/ using document root <none> and password file <none>
I20251212 08:53:26.594726 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:26.594795 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:26.594918 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:26.595546 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "6778793a8cda4894820e883bb039dcfb"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.598083 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:26.599731 4460 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.600128 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:26.600217 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "6778793a8cda4894820e883bb039dcfb"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.600351 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestGetTableSchema.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:26.619791 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:26.620327 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:26.621855 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:26.621927 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.622018 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:26.622164 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20251212 08:53:26.644591 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:41761
I20251212 08:53:26.644620 4530 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:41761 every 8 connection(s)
I20251212 08:53:26.646215 4531 heartbeater.cc:344] Connected to a master server at 127.0.208.62:39423
I20251212 08:53:26.646350 4531 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:26.646632 4531 heartbeater.cc:507] Master 127.0.208.62:39423 requested a full tablet report, sending...
I20251212 08:53:26.647408 4386 ts_manager.cc:194] Registered new tserver with Master: 6778793a8cda4894820e883bb039dcfb (127.0.208.1:41761)
I20251212 08:53:26.647753 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002317469s
I20251212 08:53:26.648491 4386 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36520
I20251212 08:53:26.658667 4386 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36542:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:26.659797 4531 heartbeater.cc:499] Master 127.0.208.62:39423 was elected leader, sending a full tablet report...
I20251212 08:53:26.672403 4491 tablet_service.cc:1505] Processing CreateTablet for tablet 849d050f3efd4666bc70ec6953443d5a (DEFAULT_TABLE table=client-testtb [id=87014d19d0b2439190ad11852d6a07b4]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:26.672483 4490 tablet_service.cc:1505] Processing CreateTablet for tablet 29be9dd765c34ffa903d9eaeb1c57a8c (DEFAULT_TABLE table=client-testtb [id=87014d19d0b2439190ad11852d6a07b4]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:26.672943 4491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 849d050f3efd4666bc70ec6953443d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:26.673169 4490 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29be9dd765c34ffa903d9eaeb1c57a8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:26.678253 4541 tablet_bootstrap.cc:492] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb: Bootstrap starting.
I20251212 08:53:26.679802 4541 tablet_bootstrap.cc:654] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:26.681607 4541 tablet_bootstrap.cc:492] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb: No bootstrap required, opened a new log
I20251212 08:53:26.681797 4541 ts_tablet_manager.cc:1403] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251212 08:53:26.682436 4541 raft_consensus.cc:359] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } }
I20251212 08:53:26.682587 4541 raft_consensus.cc:385] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:26.682669 4541 raft_consensus.cc:740] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6778793a8cda4894820e883bb039dcfb, State: Initialized, Role: FOLLOWER
I20251212 08:53:26.682891 4541 consensus_queue.cc:260] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } }
I20251212 08:53:26.683027 4541 raft_consensus.cc:399] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:26.683116 4541 raft_consensus.cc:493] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:26.683218 4541 raft_consensus.cc:3060] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:26.685709 4541 raft_consensus.cc:515] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } }
I20251212 08:53:26.685928 4541 leader_election.cc:304] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6778793a8cda4894820e883bb039dcfb; no voters:
I20251212 08:53:26.686378 4541 leader_election.cc:290] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:26.686470 4543 raft_consensus.cc:2804] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:26.686961 4543 raft_consensus.cc:697] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 1 LEADER]: Becoming Leader. State: Replica: 6778793a8cda4894820e883bb039dcfb, State: Running, Role: LEADER
I20251212 08:53:26.687136 4541 ts_tablet_manager.cc:1434] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20251212 08:53:26.687202 4543 consensus_queue.cc:237] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } }
I20251212 08:53:26.687433 4541 tablet_bootstrap.cc:492] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb: Bootstrap starting.
I20251212 08:53:26.689291 4385 catalog_manager.cc:5654] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 6778793a8cda4894820e883bb039dcfb (127.0.208.1). New cstate: current_term: 1 leader_uuid: "6778793a8cda4894820e883bb039dcfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:26.689416 4541 tablet_bootstrap.cc:654] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:26.691120 4541 tablet_bootstrap.cc:492] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb: No bootstrap required, opened a new log
I20251212 08:53:26.691845 4541 ts_tablet_manager.cc:1403] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.004s
I20251212 08:53:26.692637 4541 raft_consensus.cc:359] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } }
I20251212 08:53:26.692787 4541 raft_consensus.cc:385] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:26.692885 4541 raft_consensus.cc:740] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6778793a8cda4894820e883bb039dcfb, State: Initialized, Role: FOLLOWER
I20251212 08:53:26.693099 4541 consensus_queue.cc:260] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } }
I20251212 08:53:26.693248 4541 raft_consensus.cc:399] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:26.693343 4541 raft_consensus.cc:493] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:26.693454 4541 raft_consensus.cc:3060] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:26.695431 4541 raft_consensus.cc:515] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } }
I20251212 08:53:26.695636 4541 leader_election.cc:304] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6778793a8cda4894820e883bb039dcfb; no voters:
I20251212 08:53:26.695818 4541 leader_election.cc:290] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:26.695868 4543 raft_consensus.cc:2804] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:26.696214 4541 ts_tablet_manager.cc:1434] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251212 08:53:26.696236 4543 raft_consensus.cc:697] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 1 LEADER]: Becoming Leader. State: Replica: 6778793a8cda4894820e883bb039dcfb, State: Running, Role: LEADER
I20251212 08:53:26.696483 4543 consensus_queue.cc:237] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } }
I20251212 08:53:26.698505 4385 catalog_manager.cc:5654] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 6778793a8cda4894820e883bb039dcfb (127.0.208.1). New cstate: current_term: 1 leader_uuid: "6778793a8cda4894820e883bb039dcfb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6778793a8cda4894820e883bb039dcfb" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 41761 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:26.712395 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:26.722275 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:26.722736 832 tablet_replica.cc:333] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb: stopping tablet replica
I20251212 08:53:26.723017 832 raft_consensus.cc:2243] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:26.723237 832 raft_consensus.cc:2272] T 849d050f3efd4666bc70ec6953443d5a P 6778793a8cda4894820e883bb039dcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:26.724143 832 tablet_replica.cc:333] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb: stopping tablet replica
I20251212 08:53:26.724329 832 raft_consensus.cc:2243] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:26.724494 832 raft_consensus.cc:2272] T 29be9dd765c34ffa903d9eaeb1c57a8c P 6778793a8cda4894820e883bb039dcfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:26.739424 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:26.744244 832 master.cc:561] Master@127.0.208.62:39423 shutting down...
I20251212 08:53:26.753134 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:26.753381 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:26.753572 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 52271d97bd014e7ca1a5ae9efa243fdc: stopping tablet replica
I20251212 08:53:26.757663 832 master.cc:583] Master@127.0.208.62:39423 shutdown complete.
[ OK ] ClientTest.TestGetTableSchema (322 ms)
[ RUN ] ClientTest.TestCreateDuplicateTable
I20251212 08:53:26.790158 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:39383
I20251212 08:53:26.790683 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:26.793097 4550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:26.793118 4551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:26.793556 4553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:26.793646 832 server_base.cc:1047] running on GCE node
I20251212 08:53:26.793959 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:26.794018 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:26.794052 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529606794048 us; error 0 us; skew 500 ppm
I20251212 08:53:26.794193 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:26.795483 832 webserver.cc:492] Webserver started at http://127.0.208.62:37067/ using document root <none> and password file <none>
I20251212 08:53:26.795701 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:26.795771 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:26.795925 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:26.796588 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "8ab90e17621841989fd3ddd2be930f4c"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.799031 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251212 08:53:26.800616 4558 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.801002 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:26.801087 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "8ab90e17621841989fd3ddd2be930f4c"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.801182 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:26.831261 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:26.831835 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:26.847222 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:39383
I20251212 08:53:26.847254 4619 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:39383 every 8 connection(s)
I20251212 08:53:26.848500 4620 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:26.852810 4620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c: Bootstrap starting.
I20251212 08:53:26.854146 4620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:26.855799 4620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c: No bootstrap required, opened a new log
I20251212 08:53:26.856487 4620 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab90e17621841989fd3ddd2be930f4c" member_type: VOTER }
I20251212 08:53:26.856629 4620 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:26.856693 4620 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ab90e17621841989fd3ddd2be930f4c, State: Initialized, Role: FOLLOWER
I20251212 08:53:26.856886 4620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab90e17621841989fd3ddd2be930f4c" member_type: VOTER }
I20251212 08:53:26.856992 4620 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:26.857052 4620 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:26.857125 4620 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:26.859081 4620 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab90e17621841989fd3ddd2be930f4c" member_type: VOTER }
I20251212 08:53:26.859273 4620 leader_election.cc:304] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ab90e17621841989fd3ddd2be930f4c; no voters:
I20251212 08:53:26.859639 4620 leader_election.cc:290] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:26.859767 4623 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:26.860237 4623 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 1 LEADER]: Becoming Leader. State: Replica: 8ab90e17621841989fd3ddd2be930f4c, State: Running, Role: LEADER
I20251212 08:53:26.860481 4623 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab90e17621841989fd3ddd2be930f4c" member_type: VOTER }
I20251212 08:53:26.860495 4620 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:26.861342 4624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ab90e17621841989fd3ddd2be930f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab90e17621841989fd3ddd2be930f4c" member_type: VOTER } }
I20251212 08:53:26.861392 4625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ab90e17621841989fd3ddd2be930f4c. Latest consensus state: current_term: 1 leader_uuid: "8ab90e17621841989fd3ddd2be930f4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ab90e17621841989fd3ddd2be930f4c" member_type: VOTER } }
I20251212 08:53:26.861577 4625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:26.862035 4624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:26.863018 4629 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:26.864971 4629 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:26.865334 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:26.867746 4629 catalog_manager.cc:1357] Generated new cluster ID: 599175edef304525a3afd10b4edb2970
I20251212 08:53:26.867811 4629 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:26.881824 4629 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:26.882786 4629 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:26.896363 4629 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c: Generated new TSK 0
I20251212 08:53:26.896628 4629 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:26.898181 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:26.901000 4641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:26.901193 4644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:26.901209 4642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:26.902231 832 server_base.cc:1047] running on GCE node
I20251212 08:53:26.902474 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:26.902527 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:26.902575 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529606902571 us; error 0 us; skew 500 ppm
I20251212 08:53:26.902725 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:26.903970 832 webserver.cc:492] Webserver started at http://127.0.208.1:33455/ using document root <none> and password file <none>
I20251212 08:53:26.904178 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:26.904245 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:26.904385 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:26.905045 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "1a044b3543684a8fba9a2d4fc0547ed4"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.907677 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:26.909544 4649 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.909956 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:26.910085 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "1a044b3543684a8fba9a2d4fc0547ed4"
format_stamp: "Formatted at 2025-12-12 08:53:26 on dist-test-slave-qdpb"
I20251212 08:53:26.910193 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestCreateDuplicateTable.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:26.915309 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:26.915838 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:26.917415 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:26.917479 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.917570 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:26.917622 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:26.935187 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:46735
I20251212 08:53:26.935196 4719 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:46735 every 8 connection(s)
I20251212 08:53:26.936882 4720 heartbeater.cc:344] Connected to a master server at 127.0.208.62:39383
I20251212 08:53:26.936996 4720 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:26.937301 4720 heartbeater.cc:507] Master 127.0.208.62:39383 requested a full tablet report, sending...
I20251212 08:53:26.938050 4575 ts_manager.cc:194] Registered new tserver with Master: 1a044b3543684a8fba9a2d4fc0547ed4 (127.0.208.1:46735)
I20251212 08:53:26.938377 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002279013s
I20251212 08:53:26.939538 4575 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50842
I20251212 08:53:26.948834 4575 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50858:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:26.950958 4720 heartbeater.cc:499] Master 127.0.208.62:39383 was elected leader, sending a full tablet report...
I20251212 08:53:26.961531 4683 tablet_service.cc:1505] Processing CreateTablet for tablet 22228c1f210344f78ecb24a8ab30459c (DEFAULT_TABLE table=client-testtb [id=22f33aa1a00e424dbf7eeff0ec2fe646]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:26.961534 4681 tablet_service.cc:1505] Processing CreateTablet for tablet 1170c9ddba8943e68ba9204ea58ed630 (DEFAULT_TABLE table=client-testtb [id=22f33aa1a00e424dbf7eeff0ec2fe646]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:26.962059 4681 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1170c9ddba8943e68ba9204ea58ed630. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:26.962240 4683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22228c1f210344f78ecb24a8ab30459c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:26.967641 4730 tablet_bootstrap.cc:492] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4: Bootstrap starting.
I20251212 08:53:26.969149 4730 tablet_bootstrap.cc:654] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:26.970713 4730 tablet_bootstrap.cc:492] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4: No bootstrap required, opened a new log
I20251212 08:53:26.970821 4730 ts_tablet_manager.cc:1403] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:53:26.971376 4730 raft_consensus.cc:359] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } }
I20251212 08:53:26.971504 4730 raft_consensus.cc:385] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:26.971537 4730 raft_consensus.cc:740] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a044b3543684a8fba9a2d4fc0547ed4, State: Initialized, Role: FOLLOWER
I20251212 08:53:26.971729 4730 consensus_queue.cc:260] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } }
I20251212 08:53:26.971843 4730 raft_consensus.cc:399] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:26.971887 4730 raft_consensus.cc:493] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:26.971941 4730 raft_consensus.cc:3060] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:26.973879 4730 raft_consensus.cc:515] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } }
I20251212 08:53:26.974083 4730 leader_election.cc:304] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a044b3543684a8fba9a2d4fc0547ed4; no voters:
I20251212 08:53:26.974531 4730 leader_election.cc:290] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:26.974669 4732 raft_consensus.cc:2804] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:26.975262 4732 raft_consensus.cc:697] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 LEADER]: Becoming Leader. State: Replica: 1a044b3543684a8fba9a2d4fc0547ed4, State: Running, Role: LEADER
I20251212 08:53:26.975281 4730 ts_tablet_manager.cc:1434] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20251212 08:53:26.975486 4732 consensus_queue.cc:237] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } }
I20251212 08:53:26.975550 4730 tablet_bootstrap.cc:492] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4: Bootstrap starting.
I20251212 08:53:26.977293 4730 tablet_bootstrap.cc:654] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:26.977396 4575 catalog_manager.cc:5654] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a044b3543684a8fba9a2d4fc0547ed4 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:26.978870 4730 tablet_bootstrap.cc:492] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4: No bootstrap required, opened a new log
I20251212 08:53:26.979020 4730 ts_tablet_manager.cc:1403] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251212 08:53:26.979807 4730 raft_consensus.cc:359] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } }
I20251212 08:53:26.979971 4730 raft_consensus.cc:385] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:26.980075 4730 raft_consensus.cc:740] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a044b3543684a8fba9a2d4fc0547ed4, State: Initialized, Role: FOLLOWER
I20251212 08:53:26.980352 4730 consensus_queue.cc:260] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } }
I20251212 08:53:26.980473 4730 raft_consensus.cc:399] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:26.980532 4730 raft_consensus.cc:493] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:26.981072 4730 raft_consensus.cc:3060] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:26.983107 4730 raft_consensus.cc:515] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } }
I20251212 08:53:26.983286 4730 leader_election.cc:304] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a044b3543684a8fba9a2d4fc0547ed4; no voters:
I20251212 08:53:26.983424 4730 leader_election.cc:290] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:26.983547 4732 raft_consensus.cc:2804] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:26.983728 4732 raft_consensus.cc:697] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 LEADER]: Becoming Leader. State: Replica: 1a044b3543684a8fba9a2d4fc0547ed4, State: Running, Role: LEADER
I20251212 08:53:26.983812 4730 ts_tablet_manager.cc:1434] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251212 08:53:26.984001 4732 consensus_queue.cc:237] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } }
I20251212 08:53:26.986125 4575 catalog_manager.cc:5654] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1a044b3543684a8fba9a2d4fc0547ed4 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a044b3543684a8fba9a2d4fc0547ed4" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46735 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:26.997948 4575 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50858:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:26.999944 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:27.007946 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:27.008394 832 tablet_replica.cc:333] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4: stopping tablet replica
I20251212 08:53:27.008612 832 raft_consensus.cc:2243] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.008805 832 raft_consensus.cc:2272] T 1170c9ddba8943e68ba9204ea58ed630 P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.009680 832 tablet_replica.cc:333] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4: stopping tablet replica
I20251212 08:53:27.009827 832 raft_consensus.cc:2243] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.009989 832 raft_consensus.cc:2272] T 22228c1f210344f78ecb24a8ab30459c P 1a044b3543684a8fba9a2d4fc0547ed4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.024516 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:27.028682 832 master.cc:561] Master@127.0.208.62:39383 shutting down...
I20251212 08:53:27.037972 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.038239 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.038393 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8ab90e17621841989fd3ddd2be930f4c: stopping tablet replica
I20251212 08:53:27.053145 832 master.cc:583] Master@127.0.208.62:39383 shutdown complete.
[ OK ] ClientTest.TestCreateDuplicateTable (273 ms)
[ RUN ] ClientTest.TestInsertEmptyPK
I20251212 08:53:27.063463 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:36795
I20251212 08:53:27.063951 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:27.066339 4739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:27.066411 4740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:27.066778 4742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:27.067041 832 server_base.cc:1047] running on GCE node
I20251212 08:53:27.067371 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:27.067468 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:27.067536 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529607067516 us; error 0 us; skew 500 ppm
I20251212 08:53:27.067693 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:27.068852 832 webserver.cc:492] Webserver started at http://127.0.208.62:33741/ using document root <none> and password file <none>
I20251212 08:53:27.069079 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:27.069161 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:27.069288 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:27.069904 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "14c245b38cba48fe89984bc0591d8fca"
format_stamp: "Formatted at 2025-12-12 08:53:27 on dist-test-slave-qdpb"
I20251212 08:53:27.072319 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:53:27.073920 4747 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:27.074323 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:27.074491 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "14c245b38cba48fe89984bc0591d8fca"
format_stamp: "Formatted at 2025-12-12 08:53:27 on dist-test-slave-qdpb"
I20251212 08:53:27.074622 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:27.087944 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:27.088482 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:27.102365 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:36795
I20251212 08:53:27.102396 4808 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:36795 every 8 connection(s)
I20251212 08:53:27.103679 4809 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:27.108033 4809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca: Bootstrap starting.
I20251212 08:53:27.109364 4809 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:27.110911 4809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca: No bootstrap required, opened a new log
I20251212 08:53:27.111512 4809 raft_consensus.cc:359] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c245b38cba48fe89984bc0591d8fca" member_type: VOTER }
I20251212 08:53:27.111660 4809 raft_consensus.cc:385] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:27.111737 4809 raft_consensus.cc:740] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14c245b38cba48fe89984bc0591d8fca, State: Initialized, Role: FOLLOWER
I20251212 08:53:27.111953 4809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c245b38cba48fe89984bc0591d8fca" member_type: VOTER }
I20251212 08:53:27.112082 4809 raft_consensus.cc:399] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:27.112159 4809 raft_consensus.cc:493] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:27.112251 4809 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:27.114162 4809 raft_consensus.cc:515] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c245b38cba48fe89984bc0591d8fca" member_type: VOTER }
I20251212 08:53:27.114347 4809 leader_election.cc:304] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14c245b38cba48fe89984bc0591d8fca; no voters:
I20251212 08:53:27.114730 4809 leader_election.cc:290] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:27.114872 4812 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:27.115301 4812 raft_consensus.cc:697] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 1 LEADER]: Becoming Leader. State: Replica: 14c245b38cba48fe89984bc0591d8fca, State: Running, Role: LEADER
I20251212 08:53:27.115530 4812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c245b38cba48fe89984bc0591d8fca" member_type: VOTER }
I20251212 08:53:27.115571 4809 sys_catalog.cc:565] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:27.116256 4814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14c245b38cba48fe89984bc0591d8fca. Latest consensus state: current_term: 1 leader_uuid: "14c245b38cba48fe89984bc0591d8fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c245b38cba48fe89984bc0591d8fca" member_type: VOTER } }
I20251212 08:53:27.116348 4813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14c245b38cba48fe89984bc0591d8fca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14c245b38cba48fe89984bc0591d8fca" member_type: VOTER } }
I20251212 08:53:27.116488 4813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:27.116487 4814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:27.117101 4817 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:27.118613 4817 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:27.121366 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:27.121590 4817 catalog_manager.cc:1357] Generated new cluster ID: ef4960e97261480195777070e3154699
I20251212 08:53:27.121682 4817 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:27.134846 4817 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:27.135622 4817 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:27.145668 4817 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca: Generated new TSK 0
I20251212 08:53:27.145993 4817 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:27.154063 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:27.156741 4831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:27.156741 4830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:27.157599 4833 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:27.158795 832 server_base.cc:1047] running on GCE node
I20251212 08:53:27.159055 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:27.159121 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:27.159157 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529607159142 us; error 0 us; skew 500 ppm
I20251212 08:53:27.159296 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:27.160516 832 webserver.cc:492] Webserver started at http://127.0.208.1:39375/ using document root <none> and password file <none>
I20251212 08:53:27.160707 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:27.160777 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:27.160892 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:27.161497 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "998ab85282b046db90b8740beef76073"
format_stamp: "Formatted at 2025-12-12 08:53:27 on dist-test-slave-qdpb"
I20251212 08:53:27.163918 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:53:27.165642 4838 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:27.166014 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:27.166153 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "998ab85282b046db90b8740beef76073"
format_stamp: "Formatted at 2025-12-12 08:53:27 on dist-test-slave-qdpb"
I20251212 08:53:27.166271 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestInsertEmptyPK.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:27.182473 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:27.183065 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:27.184785 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:27.184885 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:27.185011 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:27.185122 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:27.203594 4908 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:36635 every 8 connection(s)
I20251212 08:53:27.203594 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:36635
I20251212 08:53:27.205375 4909 heartbeater.cc:344] Connected to a master server at 127.0.208.62:36795
I20251212 08:53:27.205511 4909 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:27.205857 4909 heartbeater.cc:507] Master 127.0.208.62:36795 requested a full tablet report, sending...
I20251212 08:53:27.206759 4764 ts_manager.cc:194] Registered new tserver with Master: 998ab85282b046db90b8740beef76073 (127.0.208.1:36635)
I20251212 08:53:27.206996 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002331091s
I20251212 08:53:27.207909 4764 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47014
I20251212 08:53:27.218421 4764 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47032:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:27.219995 4909 heartbeater.cc:499] Master 127.0.208.62:36795 was elected leader, sending a full tablet report...
I20251212 08:53:27.231169 4869 tablet_service.cc:1505] Processing CreateTablet for tablet b17a091a171d40bfba6c3f0d574ce7dc (DEFAULT_TABLE table=client-testtb [id=33faf686a6fb466c9fa7fca565f9de1d]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:27.231189 4868 tablet_service.cc:1505] Processing CreateTablet for tablet d9fa2db91b114809ad6937c41e1aa11e (DEFAULT_TABLE table=client-testtb [id=33faf686a6fb466c9fa7fca565f9de1d]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:27.231690 4869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b17a091a171d40bfba6c3f0d574ce7dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:27.231915 4868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d9fa2db91b114809ad6937c41e1aa11e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:27.237128 4919 tablet_bootstrap.cc:492] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073: Bootstrap starting.
I20251212 08:53:27.238862 4919 tablet_bootstrap.cc:654] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:27.240751 4919 tablet_bootstrap.cc:492] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073: No bootstrap required, opened a new log
I20251212 08:53:27.240904 4919 ts_tablet_manager.cc:1403] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:27.241528 4919 raft_consensus.cc:359] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.241669 4919 raft_consensus.cc:385] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:27.241747 4919 raft_consensus.cc:740] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998ab85282b046db90b8740beef76073, State: Initialized, Role: FOLLOWER
I20251212 08:53:27.241978 4919 consensus_queue.cc:260] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.242108 4919 raft_consensus.cc:399] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:27.242170 4919 raft_consensus.cc:493] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:27.242239 4919 raft_consensus.cc:3060] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:27.244644 4919 raft_consensus.cc:515] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.244836 4919 leader_election.cc:304] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 998ab85282b046db90b8740beef76073; no voters:
I20251212 08:53:27.245218 4919 leader_election.cc:290] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:27.245369 4921 raft_consensus.cc:2804] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:27.245822 4921 raft_consensus.cc:697] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 1 LEADER]: Becoming Leader. State: Replica: 998ab85282b046db90b8740beef76073, State: Running, Role: LEADER
I20251212 08:53:27.246353 4919 ts_tablet_manager.cc:1434] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:53:27.246347 4921 consensus_queue.cc:237] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.246690 4919 tablet_bootstrap.cc:492] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073: Bootstrap starting.
I20251212 08:53:27.248600 4919 tablet_bootstrap.cc:654] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:27.248594 4764 catalog_manager.cc:5654] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998ab85282b046db90b8740beef76073 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "998ab85282b046db90b8740beef76073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:27.255425 4919 tablet_bootstrap.cc:492] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073: No bootstrap required, opened a new log
I20251212 08:53:27.255591 4919 ts_tablet_manager.cc:1403] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20251212 08:53:27.256379 4919 raft_consensus.cc:359] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.256572 4919 raft_consensus.cc:385] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:27.256671 4919 raft_consensus.cc:740] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998ab85282b046db90b8740beef76073, State: Initialized, Role: FOLLOWER
I20251212 08:53:27.256903 4919 consensus_queue.cc:260] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.257037 4919 raft_consensus.cc:399] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:27.257117 4919 raft_consensus.cc:493] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:27.274688 4919 raft_consensus.cc:3060] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:27.278010 4919 raft_consensus.cc:515] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.278278 4919 leader_election.cc:304] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 998ab85282b046db90b8740beef76073; no voters:
I20251212 08:53:27.278481 4919 leader_election.cc:290] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:27.278676 4921 raft_consensus.cc:2804] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:27.278959 4921 raft_consensus.cc:697] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 1 LEADER]: Becoming Leader. State: Replica: 998ab85282b046db90b8740beef76073, State: Running, Role: LEADER
I20251212 08:53:27.279129 4919 ts_tablet_manager.cc:1434] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073: Time spent starting tablet: real 0.023s user 0.021s sys 0.000s
I20251212 08:53:27.279197 4921 consensus_queue.cc:237] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.281611 4763 catalog_manager.cc:5654] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998ab85282b046db90b8740beef76073 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "998ab85282b046db90b8740beef76073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:27.293438 4763 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47032:
name: "empty-pk"
schema {
columns {
name: "k1"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "v1"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "k1"
}
}
}
I20251212 08:53:27.300940 4869 tablet_service.cc:1505] Processing CreateTablet for tablet 3142a038128b475a867407c6f31eb011 (DEFAULT_TABLE table=empty-pk [id=bb1137075b344eecb1f03879de2dafc7]), partition=RANGE (k1) PARTITION UNBOUNDED
I20251212 08:53:27.301390 4869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3142a038128b475a867407c6f31eb011. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:27.306980 4919 tablet_bootstrap.cc:492] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073: Bootstrap starting.
I20251212 08:53:27.308435 4919 tablet_bootstrap.cc:654] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:27.310117 4919 tablet_bootstrap.cc:492] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073: No bootstrap required, opened a new log
I20251212 08:53:27.310252 4919 ts_tablet_manager.cc:1403] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:53:27.310930 4919 raft_consensus.cc:359] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.311062 4919 raft_consensus.cc:385] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:27.311115 4919 raft_consensus.cc:740] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 998ab85282b046db90b8740beef76073, State: Initialized, Role: FOLLOWER
I20251212 08:53:27.311288 4919 consensus_queue.cc:260] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.311388 4919 raft_consensus.cc:399] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:27.311446 4919 raft_consensus.cc:493] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:27.311519 4919 raft_consensus.cc:3060] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:27.313474 4919 raft_consensus.cc:515] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.313652 4919 leader_election.cc:304] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 998ab85282b046db90b8740beef76073; no voters:
I20251212 08:53:27.313822 4919 leader_election.cc:290] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:27.313908 4921 raft_consensus.cc:2804] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:27.314103 4921 raft_consensus.cc:697] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 1 LEADER]: Becoming Leader. State: Replica: 998ab85282b046db90b8740beef76073, State: Running, Role: LEADER
I20251212 08:53:27.314308 4919 ts_tablet_manager.cc:1434] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073: Time spent starting tablet: real 0.004s user 0.003s sys 0.002s
I20251212 08:53:27.314337 4921 consensus_queue.cc:237] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } }
I20251212 08:53:27.316334 4764 catalog_manager.cc:5654] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 reported cstate change: term changed from 0 to 1, leader changed from <none> to 998ab85282b046db90b8740beef76073 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "998ab85282b046db90b8740beef76073" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "998ab85282b046db90b8740beef76073" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36635 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:27.443044 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:27.453430 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:27.453812 832 tablet_replica.cc:333] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073: stopping tablet replica
I20251212 08:53:27.454028 832 raft_consensus.cc:2243] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.454229 832 raft_consensus.cc:2272] T 3142a038128b475a867407c6f31eb011 P 998ab85282b046db90b8740beef76073 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.455161 832 tablet_replica.cc:333] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073: stopping tablet replica
I20251212 08:53:27.455297 832 raft_consensus.cc:2243] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.455426 832 raft_consensus.cc:2272] T b17a091a171d40bfba6c3f0d574ce7dc P 998ab85282b046db90b8740beef76073 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.456136 832 tablet_replica.cc:333] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073: stopping tablet replica
I20251212 08:53:27.456260 832 raft_consensus.cc:2243] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.456420 832 raft_consensus.cc:2272] T d9fa2db91b114809ad6937c41e1aa11e P 998ab85282b046db90b8740beef76073 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.471565 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:27.477207 832 master.cc:561] Master@127.0.208.62:36795 shutting down...
I20251212 08:53:27.488323 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.488603 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.488768 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 14c245b38cba48fe89984bc0591d8fca: stopping tablet replica
I20251212 08:53:27.503800 832 master.cc:583] Master@127.0.208.62:36795 shutdown complete.
[ OK ] ClientTest.TestInsertEmptyPK (450 ms)
[ RUN ] ClientTest.TestClonePredicates
I20251212 08:53:27.514546 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:36835
I20251212 08:53:27.515106 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:27.517478 4937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:27.517494 4938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:27.518335 4940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:27.518357 832 server_base.cc:1047] running on GCE node
I20251212 08:53:27.518711 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:27.518762 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:27.518795 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529607518790 us; error 0 us; skew 500 ppm
I20251212 08:53:27.518929 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:27.520077 832 webserver.cc:492] Webserver started at http://127.0.208.62:35043/ using document root <none> and password file <none>
I20251212 08:53:27.520262 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:27.520356 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:27.520469 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:27.521065 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "9cff809b97fd403f9e8be8066a92607e"
format_stamp: "Formatted at 2025-12-12 08:53:27 on dist-test-slave-qdpb"
I20251212 08:53:27.523388 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:53:27.524895 4945 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:27.525239 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:27.525324 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "9cff809b97fd403f9e8be8066a92607e"
format_stamp: "Formatted at 2025-12-12 08:53:27 on dist-test-slave-qdpb"
I20251212 08:53:27.525429 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:27.541898 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:27.542416 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:27.558854 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:36835
I20251212 08:53:27.558854 5006 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:36835 every 8 connection(s)
I20251212 08:53:27.560312 5007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:27.564754 5007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e: Bootstrap starting.
I20251212 08:53:27.565905 5007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:27.567371 5007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e: No bootstrap required, opened a new log
I20251212 08:53:27.567987 5007 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cff809b97fd403f9e8be8066a92607e" member_type: VOTER }
I20251212 08:53:27.568110 5007 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:27.568161 5007 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9cff809b97fd403f9e8be8066a92607e, State: Initialized, Role: FOLLOWER
I20251212 08:53:27.568373 5007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cff809b97fd403f9e8be8066a92607e" member_type: VOTER }
I20251212 08:53:27.568475 5007 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:27.568526 5007 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:27.568600 5007 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:27.570417 5007 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cff809b97fd403f9e8be8066a92607e" member_type: VOTER }
I20251212 08:53:27.570582 5007 leader_election.cc:304] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9cff809b97fd403f9e8be8066a92607e; no voters:
I20251212 08:53:27.570996 5007 leader_election.cc:290] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:27.571123 5010 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:27.571514 5010 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 1 LEADER]: Becoming Leader. State: Replica: 9cff809b97fd403f9e8be8066a92607e, State: Running, Role: LEADER
I20251212 08:53:27.571836 5010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cff809b97fd403f9e8be8066a92607e" member_type: VOTER }
I20251212 08:53:27.573114 5007 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:27.575380 5012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9cff809b97fd403f9e8be8066a92607e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cff809b97fd403f9e8be8066a92607e" member_type: VOTER } }
I20251212 08:53:27.575570 5012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:27.576073 5011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9cff809b97fd403f9e8be8066a92607e. Latest consensus state: current_term: 1 leader_uuid: "9cff809b97fd403f9e8be8066a92607e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9cff809b97fd403f9e8be8066a92607e" member_type: VOTER } }
I20251212 08:53:27.576227 5011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:27.583038 5017 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:27.584971 5017 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:27.585988 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:27.588522 5017 catalog_manager.cc:1357] Generated new cluster ID: b17f856d8a934945a8b63f83f6dc0211
I20251212 08:53:27.588605 5017 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:27.604736 5017 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:27.605571 5017 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:27.618592 5017 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e: Generated new TSK 0
I20251212 08:53:27.618867 5017 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:27.651324 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:27.654850 5030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:27.654906 5029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:27.655300 5032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:27.655333 832 server_base.cc:1047] running on GCE node
I20251212 08:53:27.655658 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:27.655715 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:27.655753 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529607655749 us; error 0 us; skew 500 ppm
I20251212 08:53:27.655879 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:27.657153 832 webserver.cc:492] Webserver started at http://127.0.208.1:41009/ using document root <none> and password file <none>
I20251212 08:53:27.657343 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:27.657409 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:27.657524 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:27.658162 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "9b235adde52840c1958df7635a15764d"
format_stamp: "Formatted at 2025-12-12 08:53:27 on dist-test-slave-qdpb"
I20251212 08:53:27.660606 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:53:27.662200 5037 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:27.662536 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:27.662622 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "9b235adde52840c1958df7635a15764d"
format_stamp: "Formatted at 2025-12-12 08:53:27 on dist-test-slave-qdpb"
I20251212 08:53:27.662726 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestClonePredicates.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:27.683715 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:27.684324 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:27.688434 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:27.688619 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:27.688740 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:27.688797 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:27.721853 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:36341
I20251212 08:53:27.721870 5107 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:36341 every 8 connection(s)
I20251212 08:53:27.723670 5108 heartbeater.cc:344] Connected to a master server at 127.0.208.62:36835
I20251212 08:53:27.723802 5108 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:27.724143 5108 heartbeater.cc:507] Master 127.0.208.62:36835 requested a full tablet report, sending...
I20251212 08:53:27.724898 4962 ts_manager.cc:194] Registered new tserver with Master: 9b235adde52840c1958df7635a15764d (127.0.208.1:36341)
I20251212 08:53:27.725157 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002267001s
I20251212 08:53:27.726408 4962 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46420
I20251212 08:53:27.737488 5108 heartbeater.cc:499] Master 127.0.208.62:36835 was elected leader, sending a full tablet report...
I20251212 08:53:27.739100 4962 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46444:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:27.753386 5069 tablet_service.cc:1505] Processing CreateTablet for tablet d7f4e9a88d10493b82c6932afa762ae4 (DEFAULT_TABLE table=client-testtb [id=56c265f056294610a91df1072ca806e5]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:27.753475 5073 tablet_service.cc:1505] Processing CreateTablet for tablet 0bcde0bf83af4d209fd77aa9317d9615 (DEFAULT_TABLE table=client-testtb [id=56c265f056294610a91df1072ca806e5]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:27.753890 5069 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f4e9a88d10493b82c6932afa762ae4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:27.754132 5073 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0bcde0bf83af4d209fd77aa9317d9615. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:27.761214 5118 tablet_bootstrap.cc:492] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d: Bootstrap starting.
I20251212 08:53:27.762576 5118 tablet_bootstrap.cc:654] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:27.764425 5118 tablet_bootstrap.cc:492] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d: No bootstrap required, opened a new log
I20251212 08:53:27.764561 5118 ts_tablet_manager.cc:1403] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251212 08:53:27.765161 5118 raft_consensus.cc:359] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } }
I20251212 08:53:27.765305 5118 raft_consensus.cc:385] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:27.765376 5118 raft_consensus.cc:740] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b235adde52840c1958df7635a15764d, State: Initialized, Role: FOLLOWER
I20251212 08:53:27.765587 5118 consensus_queue.cc:260] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } }
I20251212 08:53:27.765723 5118 raft_consensus.cc:399] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:27.765795 5118 raft_consensus.cc:493] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:27.765878 5118 raft_consensus.cc:3060] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:27.767833 5118 raft_consensus.cc:515] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } }
I20251212 08:53:27.768036 5118 leader_election.cc:304] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b235adde52840c1958df7635a15764d; no voters:
I20251212 08:53:27.768458 5118 leader_election.cc:290] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:27.768568 5120 raft_consensus.cc:2804] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:27.769215 5118 ts_tablet_manager.cc:1434] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20251212 08:53:27.769237 5120 raft_consensus.cc:697] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 1 LEADER]: Becoming Leader. State: Replica: 9b235adde52840c1958df7635a15764d, State: Running, Role: LEADER
I20251212 08:53:27.769479 5120 consensus_queue.cc:237] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } }
I20251212 08:53:27.769526 5118 tablet_bootstrap.cc:492] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d: Bootstrap starting.
I20251212 08:53:27.771556 5118 tablet_bootstrap.cc:654] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:27.771843 4962 catalog_manager.cc:5654] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b235adde52840c1958df7635a15764d (127.0.208.1). New cstate: current_term: 1 leader_uuid: "9b235adde52840c1958df7635a15764d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:27.773726 5118 tablet_bootstrap.cc:492] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d: No bootstrap required, opened a new log
I20251212 08:53:27.773867 5118 ts_tablet_manager.cc:1403] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251212 08:53:27.774556 5118 raft_consensus.cc:359] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } }
I20251212 08:53:27.774784 5118 raft_consensus.cc:385] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:27.774858 5118 raft_consensus.cc:740] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b235adde52840c1958df7635a15764d, State: Initialized, Role: FOLLOWER
I20251212 08:53:27.775068 5118 consensus_queue.cc:260] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } }
I20251212 08:53:27.775192 5118 raft_consensus.cc:399] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:27.775265 5118 raft_consensus.cc:493] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:27.775347 5118 raft_consensus.cc:3060] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:27.778288 5118 raft_consensus.cc:515] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } }
I20251212 08:53:27.778515 5118 leader_election.cc:304] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b235adde52840c1958df7635a15764d; no voters:
I20251212 08:53:27.778784 5118 leader_election.cc:290] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:27.778848 5120 raft_consensus.cc:2804] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:27.779007 5120 raft_consensus.cc:697] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 1 LEADER]: Becoming Leader. State: Replica: 9b235adde52840c1958df7635a15764d, State: Running, Role: LEADER
I20251212 08:53:27.779217 5120 consensus_queue.cc:237] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } }
I20251212 08:53:27.779314 5118 ts_tablet_manager.cc:1434] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20251212 08:53:27.781052 4961 catalog_manager.cc:5654] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9b235adde52840c1958df7635a15764d (127.0.208.1). New cstate: current_term: 1 leader_uuid: "9b235adde52840c1958df7635a15764d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b235adde52840c1958df7635a15764d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36341 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:27.816321 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:27.831030 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:27.831384 832 tablet_replica.cc:333] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d: stopping tablet replica
I20251212 08:53:27.831615 832 raft_consensus.cc:2243] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.831843 832 raft_consensus.cc:2272] T d7f4e9a88d10493b82c6932afa762ae4 P 9b235adde52840c1958df7635a15764d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.832757 832 tablet_replica.cc:333] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d: stopping tablet replica
I20251212 08:53:27.832909 832 raft_consensus.cc:2243] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.833117 832 raft_consensus.cc:2272] T 0bcde0bf83af4d209fd77aa9317d9615 P 9b235adde52840c1958df7635a15764d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.849020 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:27.854089 832 master.cc:561] Master@127.0.208.62:36835 shutting down...
I20251212 08:53:27.867065 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:27.867386 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:27.867569 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9cff809b97fd403f9e8be8066a92607e: stopping tablet replica
I20251212 08:53:28.167775 832 master.cc:583] Master@127.0.208.62:36835 shutdown complete.
[ OK ] ClientTest.TestClonePredicates (665 ms)
[ RUN ] ClientTest.TestErrorCollector
I20251212 08:53:28.180183 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:33011
I20251212 08:53:28.180760 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:28.183197 5129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.183312 5130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:28.184085 832 server_base.cc:1047] running on GCE node
W20251212 08:53:28.184345 5132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:28.184651 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:28.184729 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:28.184767 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529608184763 us; error 0 us; skew 500 ppm
I20251212 08:53:28.184906 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:28.186185 832 webserver.cc:492] Webserver started at http://127.0.208.62:34399/ using document root <none> and password file <none>
I20251212 08:53:28.186384 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:28.186443 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:28.186551 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:28.187211 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "c3105e82627d4281b4e000a129364835"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.189816 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:28.191520 5137 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.191956 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:28.192047 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "c3105e82627d4281b4e000a129364835"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.192149 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:28.201341 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:28.201828 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:28.215678 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:33011
I20251212 08:53:28.215749 5198 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:33011 every 8 connection(s)
I20251212 08:53:28.217051 5199 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:28.222139 5199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835: Bootstrap starting.
I20251212 08:53:28.223367 5199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:28.224937 5199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835: No bootstrap required, opened a new log
I20251212 08:53:28.225543 5199 raft_consensus.cc:359] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3105e82627d4281b4e000a129364835" member_type: VOTER }
I20251212 08:53:28.225668 5199 raft_consensus.cc:385] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:28.225723 5199 raft_consensus.cc:740] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3105e82627d4281b4e000a129364835, State: Initialized, Role: FOLLOWER
I20251212 08:53:28.225960 5199 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3105e82627d4281b4e000a129364835" member_type: VOTER }
I20251212 08:53:28.226071 5199 raft_consensus.cc:399] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:28.226135 5199 raft_consensus.cc:493] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:28.226214 5199 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:28.228441 5199 raft_consensus.cc:515] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3105e82627d4281b4e000a129364835" member_type: VOTER }
I20251212 08:53:28.228636 5199 leader_election.cc:304] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3105e82627d4281b4e000a129364835; no voters:
I20251212 08:53:28.229012 5199 leader_election.cc:290] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:28.229123 5202 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:28.229573 5202 raft_consensus.cc:697] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 1 LEADER]: Becoming Leader. State: Replica: c3105e82627d4281b4e000a129364835, State: Running, Role: LEADER
I20251212 08:53:28.229818 5199 sys_catalog.cc:565] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:28.229818 5202 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3105e82627d4281b4e000a129364835" member_type: VOTER }
I20251212 08:53:28.231035 5203 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3105e82627d4281b4e000a129364835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3105e82627d4281b4e000a129364835" member_type: VOTER } }
I20251212 08:53:28.231202 5203 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:28.231014 5204 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3105e82627d4281b4e000a129364835. Latest consensus state: current_term: 1 leader_uuid: "c3105e82627d4281b4e000a129364835" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3105e82627d4281b4e000a129364835" member_type: VOTER } }
I20251212 08:53:28.231771 5204 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:28.232210 5208 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:28.234218 5208 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:28.235229 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:28.237358 5208 catalog_manager.cc:1357] Generated new cluster ID: cf9a89b6fc3b42b895eaac8281e61f7b
I20251212 08:53:28.237438 5208 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:28.262588 5208 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:28.263461 5208 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:28.272578 5208 catalog_manager.cc:6027] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835: Generated new TSK 0
I20251212 08:53:28.272840 5208 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:28.300271 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:28.302871 5220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.303165 5221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.303544 5223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:28.303974 832 server_base.cc:1047] running on GCE node
I20251212 08:53:28.304214 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:28.304268 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:28.304340 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529608304335 us; error 0 us; skew 500 ppm
I20251212 08:53:28.304467 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:28.305631 832 webserver.cc:492] Webserver started at http://127.0.208.1:35325/ using document root <none> and password file <none>
I20251212 08:53:28.305816 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:28.305873 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:28.305984 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:28.306604 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "135be8405f4e4c79b68d0c62861e44a8"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.309054 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20251212 08:53:28.310525 5228 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.310887 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:28.310971 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "135be8405f4e4c79b68d0c62861e44a8"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.311062 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestErrorCollector.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:28.338675 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:28.339145 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:28.340961 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:28.341046 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.341148 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:28.341193 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.359299 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:36257
I20251212 08:53:28.359313 5298 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:36257 every 8 connection(s)
I20251212 08:53:28.361064 5299 heartbeater.cc:344] Connected to a master server at 127.0.208.62:33011
I20251212 08:53:28.361187 5299 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:28.361609 5299 heartbeater.cc:507] Master 127.0.208.62:33011 requested a full tablet report, sending...
I20251212 08:53:28.362421 5154 ts_manager.cc:194] Registered new tserver with Master: 135be8405f4e4c79b68d0c62861e44a8 (127.0.208.1:36257)
I20251212 08:53:28.362763 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002304945s
I20251212 08:53:28.363641 5154 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53990
I20251212 08:53:28.374284 5154 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54014:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:28.375257 5299 heartbeater.cc:499] Master 127.0.208.62:33011 was elected leader, sending a full tablet report...
I20251212 08:53:28.389191 5258 tablet_service.cc:1505] Processing CreateTablet for tablet da5ae03903734cbf9c79a54c1fe09a72 (DEFAULT_TABLE table=client-testtb [id=447bc88c8b964fad8afd76dc9567cacf]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:28.389191 5259 tablet_service.cc:1505] Processing CreateTablet for tablet a53ec8b321a14872af3c5007e9062da7 (DEFAULT_TABLE table=client-testtb [id=447bc88c8b964fad8afd76dc9567cacf]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:28.389626 5258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet da5ae03903734cbf9c79a54c1fe09a72. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:28.389827 5259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a53ec8b321a14872af3c5007e9062da7. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:28.395387 5309 tablet_bootstrap.cc:492] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8: Bootstrap starting.
I20251212 08:53:28.396919 5309 tablet_bootstrap.cc:654] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:28.398430 5309 tablet_bootstrap.cc:492] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8: No bootstrap required, opened a new log
I20251212 08:53:28.398548 5309 ts_tablet_manager.cc:1403] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:53:28.399237 5309 raft_consensus.cc:359] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } }
I20251212 08:53:28.399461 5309 raft_consensus.cc:385] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:28.399539 5309 raft_consensus.cc:740] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135be8405f4e4c79b68d0c62861e44a8, State: Initialized, Role: FOLLOWER
I20251212 08:53:28.399736 5309 consensus_queue.cc:260] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } }
I20251212 08:53:28.399849 5309 raft_consensus.cc:399] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:28.399892 5309 raft_consensus.cc:493] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:28.399950 5309 raft_consensus.cc:3060] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:28.402212 5309 raft_consensus.cc:515] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } }
I20251212 08:53:28.402375 5309 leader_election.cc:304] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 135be8405f4e4c79b68d0c62861e44a8; no voters:
I20251212 08:53:28.402711 5309 leader_election.cc:290] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:28.402834 5311 raft_consensus.cc:2804] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:28.403267 5311 raft_consensus.cc:697] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 LEADER]: Becoming Leader. State: Replica: 135be8405f4e4c79b68d0c62861e44a8, State: Running, Role: LEADER
I20251212 08:53:28.403326 5309 ts_tablet_manager.cc:1434] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20251212 08:53:28.403553 5309 tablet_bootstrap.cc:492] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8: Bootstrap starting.
I20251212 08:53:28.403498 5311 consensus_queue.cc:237] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } }
I20251212 08:53:28.405148 5309 tablet_bootstrap.cc:654] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:28.405575 5154 catalog_manager.cc:5654] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 135be8405f4e4c79b68d0c62861e44a8 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "135be8405f4e4c79b68d0c62861e44a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:28.406699 5309 tablet_bootstrap.cc:492] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8: No bootstrap required, opened a new log
I20251212 08:53:28.406816 5309 ts_tablet_manager.cc:1403] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:28.407507 5309 raft_consensus.cc:359] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } }
I20251212 08:53:28.407640 5309 raft_consensus.cc:385] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:28.407709 5309 raft_consensus.cc:740] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135be8405f4e4c79b68d0c62861e44a8, State: Initialized, Role: FOLLOWER
I20251212 08:53:28.407974 5309 consensus_queue.cc:260] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } }
I20251212 08:53:28.408139 5309 raft_consensus.cc:399] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:28.408214 5309 raft_consensus.cc:493] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:28.408334 5309 raft_consensus.cc:3060] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:28.411307 5309 raft_consensus.cc:515] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } }
I20251212 08:53:28.411481 5309 leader_election.cc:304] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 135be8405f4e4c79b68d0c62861e44a8; no voters:
I20251212 08:53:28.411649 5309 leader_election.cc:290] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:28.411782 5311 raft_consensus.cc:2804] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:28.411993 5311 raft_consensus.cc:697] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 LEADER]: Becoming Leader. State: Replica: 135be8405f4e4c79b68d0c62861e44a8, State: Running, Role: LEADER
I20251212 08:53:28.412161 5309 ts_tablet_manager.cc:1434] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20251212 08:53:28.412228 5311 consensus_queue.cc:237] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } }
I20251212 08:53:28.413936 5154 catalog_manager.cc:5654] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 135be8405f4e4c79b68d0c62861e44a8 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "135be8405f4e4c79b68d0c62861e44a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135be8405f4e4c79b68d0c62861e44a8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36257 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:28.440454 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:28.450011 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:28.450384 832 tablet_replica.cc:333] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8: stopping tablet replica
I20251212 08:53:28.450573 832 raft_consensus.cc:2243] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:28.450768 832 raft_consensus.cc:2272] T a53ec8b321a14872af3c5007e9062da7 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:28.451611 832 tablet_replica.cc:333] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8: stopping tablet replica
I20251212 08:53:28.451716 832 raft_consensus.cc:2243] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:28.451831 832 raft_consensus.cc:2272] T da5ae03903734cbf9c79a54c1fe09a72 P 135be8405f4e4c79b68d0c62861e44a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:28.460759 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:28.465138 832 master.cc:561] Master@127.0.208.62:33011 shutting down...
I20251212 08:53:28.473739 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:28.473989 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:28.474197 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P c3105e82627d4281b4e000a129364835: stopping tablet replica
I20251212 08:53:28.478722 832 master.cc:583] Master@127.0.208.62:33011 shutdown complete.
[ OK ] ClientTest.TestErrorCollector (309 ms)
[ RUN ] ClientTest.TestBlockScannerHijackingAttempts
I20251212 08:53:28.489883 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:38535
I20251212 08:53:28.490351 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:28.492522 5320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.492712 5321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.493158 5323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:28.493795 832 server_base.cc:1047] running on GCE node
I20251212 08:53:28.493986 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:28.494031 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:28.494069 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529608494065 us; error 0 us; skew 500 ppm
I20251212 08:53:28.494211 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:28.495338 832 webserver.cc:492] Webserver started at http://127.0.208.62:33063/ using document root <none> and password file <none>
I20251212 08:53:28.495553 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:28.495604 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:28.495752 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:28.496532 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "513456cf96614495a7c46e42b3a33e55"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.498979 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:28.500545 5328 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.500855 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:53:28.500938 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "513456cf96614495a7c46e42b3a33e55"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.501036 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:28.521723 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:28.522274 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:28.537115 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:38535
I20251212 08:53:28.537153 5389 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:38535 every 8 connection(s)
I20251212 08:53:28.538836 5390 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:28.543951 5390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55: Bootstrap starting.
I20251212 08:53:28.545226 5390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:28.546806 5390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55: No bootstrap required, opened a new log
I20251212 08:53:28.547395 5390 raft_consensus.cc:359] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER }
I20251212 08:53:28.547521 5390 raft_consensus.cc:385] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:28.547556 5390 raft_consensus.cc:740] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513456cf96614495a7c46e42b3a33e55, State: Initialized, Role: FOLLOWER
I20251212 08:53:28.547775 5390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER }
I20251212 08:53:28.547874 5390 raft_consensus.cc:399] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:28.547930 5390 raft_consensus.cc:493] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:28.548005 5390 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:28.549988 5390 raft_consensus.cc:515] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER }
I20251212 08:53:28.550155 5390 leader_election.cc:304] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 513456cf96614495a7c46e42b3a33e55; no voters:
I20251212 08:53:28.550539 5390 leader_election.cc:290] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:28.550657 5393 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:28.551293 5390 sys_catalog.cc:565] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:28.551551 5393 raft_consensus.cc:697] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 LEADER]: Becoming Leader. State: Replica: 513456cf96614495a7c46e42b3a33e55, State: Running, Role: LEADER
I20251212 08:53:28.551775 5393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER }
I20251212 08:53:28.552681 5394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "513456cf96614495a7c46e42b3a33e55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER } }
I20251212 08:53:28.552714 5395 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 513456cf96614495a7c46e42b3a33e55. Latest consensus state: current_term: 1 leader_uuid: "513456cf96614495a7c46e42b3a33e55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER } }
I20251212 08:53:28.552841 5394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:28.552856 5395 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:28.553758 5401 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:28.555694 5401 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:28.556018 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:28.558629 5401 catalog_manager.cc:1357] Generated new cluster ID: a4943d5ef5584bda92652fd5ecf97b63
I20251212 08:53:28.558750 5401 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:28.585409 5401 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:28.586256 5401 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:28.596515 5401 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55: Generated new TSK 0
I20251212 08:53:28.596767 5401 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:28.621345 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:28.624071 5411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.624176 5412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.624534 5414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:28.624933 832 server_base.cc:1047] running on GCE node
I20251212 08:53:28.631695 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:28.631819 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:28.631844 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529608631841 us; error 0 us; skew 500 ppm
I20251212 08:53:28.631953 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:28.633226 832 webserver.cc:492] Webserver started at http://127.0.208.1:36297/ using document root <none> and password file <none>
I20251212 08:53:28.633407 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:28.633460 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:28.633551 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:28.634176 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "432b7b164a0e4d21b8f76b17a230d1cf"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.636610 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251212 08:53:28.638232 5419 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.638556 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:28.638649 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "432b7b164a0e4d21b8f76b17a230d1cf"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.638751 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:28.645344 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:28.645825 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:28.647368 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:28.647435 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.647518 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:28.647575 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.667574 5489 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:38381 every 8 connection(s)
I20251212 08:53:28.667574 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:38381
I20251212 08:53:28.669747 5490 heartbeater.cc:344] Connected to a master server at 127.0.208.62:38535
I20251212 08:53:28.669872 5490 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:28.670186 5490 heartbeater.cc:507] Master 127.0.208.62:38535 requested a full tablet report, sending...
I20251212 08:53:28.671002 5345 ts_manager.cc:194] Registered new tserver with Master: 432b7b164a0e4d21b8f76b17a230d1cf (127.0.208.1:38381)
I20251212 08:53:28.671249 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.00230384s
I20251212 08:53:28.672112 5345 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36468
I20251212 08:53:28.682163 5345 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36490:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:28.683714 5490 heartbeater.cc:499] Master 127.0.208.62:38535 was elected leader, sending a full tablet report...
I20251212 08:53:28.695359 5455 tablet_service.cc:1505] Processing CreateTablet for tablet 1e3079590e0446b687d76a5a3d8d520c (DEFAULT_TABLE table=client-testtb [id=17e639d5a7b84efcbd103c28c41dc4e1]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:28.695359 5454 tablet_service.cc:1505] Processing CreateTablet for tablet 2971825ac10a42a3b5dcd2a2911b519b (DEFAULT_TABLE table=client-testtb [id=17e639d5a7b84efcbd103c28c41dc4e1]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:28.695880 5454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2971825ac10a42a3b5dcd2a2911b519b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:28.696118 5455 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e3079590e0446b687d76a5a3d8d520c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:28.702693 5500 tablet_bootstrap.cc:492] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: Bootstrap starting.
I20251212 08:53:28.703969 5500 tablet_bootstrap.cc:654] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:28.705540 5500 tablet_bootstrap.cc:492] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: No bootstrap required, opened a new log
I20251212 08:53:28.705715 5500 ts_tablet_manager.cc:1403] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:28.706254 5500 raft_consensus.cc:359] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.706393 5500 raft_consensus.cc:385] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:28.706454 5500 raft_consensus.cc:740] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432b7b164a0e4d21b8f76b17a230d1cf, State: Initialized, Role: FOLLOWER
I20251212 08:53:28.706660 5500 consensus_queue.cc:260] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.706804 5500 raft_consensus.cc:399] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:28.706876 5500 raft_consensus.cc:493] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:28.706959 5500 raft_consensus.cc:3060] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:28.709476 5500 raft_consensus.cc:515] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.709666 5500 leader_election.cc:304] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 432b7b164a0e4d21b8f76b17a230d1cf; no voters:
I20251212 08:53:28.710072 5500 leader_election.cc:290] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:28.710228 5502 raft_consensus.cc:2804] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:28.710722 5502 raft_consensus.cc:697] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 LEADER]: Becoming Leader. State: Replica: 432b7b164a0e4d21b8f76b17a230d1cf, State: Running, Role: LEADER
I20251212 08:53:28.710739 5500 ts_tablet_manager.cc:1434] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: Time spent starting tablet: real 0.005s user 0.001s sys 0.004s
I20251212 08:53:28.711023 5500 tablet_bootstrap.cc:492] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: Bootstrap starting.
I20251212 08:53:28.710994 5502 consensus_queue.cc:237] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.712618 5500 tablet_bootstrap.cc:654] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:28.713254 5344 catalog_manager.cc:5654] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 432b7b164a0e4d21b8f76b17a230d1cf (127.0.208.1). New cstate: current_term: 1 leader_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:28.714632 5500 tablet_bootstrap.cc:492] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: No bootstrap required, opened a new log
I20251212 08:53:28.714740 5500 ts_tablet_manager.cc:1403] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:28.715297 5500 raft_consensus.cc:359] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.715405 5500 raft_consensus.cc:385] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:28.715485 5500 raft_consensus.cc:740] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432b7b164a0e4d21b8f76b17a230d1cf, State: Initialized, Role: FOLLOWER
I20251212 08:53:28.715750 5500 consensus_queue.cc:260] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.715898 5500 raft_consensus.cc:399] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:28.715956 5500 raft_consensus.cc:493] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:28.716038 5500 raft_consensus.cc:3060] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:28.718727 5500 raft_consensus.cc:515] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.718907 5500 leader_election.cc:304] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 432b7b164a0e4d21b8f76b17a230d1cf; no voters:
I20251212 08:53:28.719048 5500 leader_election.cc:290] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:28.719133 5502 raft_consensus.cc:2804] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:28.719383 5502 raft_consensus.cc:697] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 LEADER]: Becoming Leader. State: Replica: 432b7b164a0e4d21b8f76b17a230d1cf, State: Running, Role: LEADER
I20251212 08:53:28.719415 5500 ts_tablet_manager.cc:1434] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251212 08:53:28.719585 5502 consensus_queue.cc:237] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.721527 5344 catalog_manager.cc:5654] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 432b7b164a0e4d21b8f76b17a230d1cf (127.0.208.1). New cstate: current_term: 1 leader_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:28.731091 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:28.739912 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:28.740273 832 tablet_replica.cc:333] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: stopping tablet replica
I20251212 08:53:28.740487 832 raft_consensus.cc:2243] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:28.740648 832 raft_consensus.cc:2272] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:28.741461 832 tablet_replica.cc:333] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: stopping tablet replica
I20251212 08:53:28.741572 832 raft_consensus.cc:2243] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:28.741699 832 raft_consensus.cc:2272] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:28.746042 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:28.750806 832 master.cc:561] Master@127.0.208.62:38535 shutting down...
I20251212 08:53:28.760531 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:28.760795 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:28.760980 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55: stopping tablet replica
I20251212 08:53:28.765302 832 master.cc:583] Master@127.0.208.62:38535 shutdown complete.
I20251212 08:53:28.780004 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:28.782727 5509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.784132 5510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.784548 5512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:28.784835 832 server_base.cc:1047] running on GCE node
I20251212 08:53:28.785094 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:28.785151 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:28.785173 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529608785170 us; error 0 us; skew 500 ppm
I20251212 08:53:28.785315 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:28.786527 832 webserver.cc:492] Webserver started at http://127.0.208.62:44831/ using document root <none> and password file <none>
I20251212 08:53:28.786736 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:28.786788 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:28.790705 832 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.001s sys 0.001s
I20251212 08:53:28.791993 5517 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.792387 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:28.792470 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "513456cf96614495a7c46e42b3a33e55"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.792570 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:28.799868 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:28.800362 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:28.812844 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:38535
I20251212 08:53:28.812862 5578 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:38535 every 8 connection(s)
I20251212 08:53:28.816540 5579 sys_catalog.cc:263] Verifying existing consensus state
I20251212 08:53:28.818430 5579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55: Bootstrap starting.
I20251212 08:53:28.828758 5579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=6 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:28.829279 5579 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55: Bootstrap complete.
I20251212 08:53:28.830063 5579 raft_consensus.cc:359] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER }
I20251212 08:53:28.830195 5579 raft_consensus.cc:740] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513456cf96614495a7c46e42b3a33e55, State: Initialized, Role: FOLLOWER
I20251212 08:53:28.830387 5579 consensus_queue.cc:260] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER }
I20251212 08:53:28.830492 5579 raft_consensus.cc:399] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:28.830538 5579 raft_consensus.cc:493] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:28.830613 5579 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 1 FOLLOWER]: Advancing to term 2
I20251212 08:53:28.832721 5579 raft_consensus.cc:515] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER }
I20251212 08:53:28.832901 5579 leader_election.cc:304] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 513456cf96614495a7c46e42b3a33e55; no voters:
I20251212 08:53:28.833326 5579 leader_election.cc:290] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [CANDIDATE]: Term 2 election: Requested vote from peers
I20251212 08:53:28.833442 5582 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 2 FOLLOWER]: Leader election won for term 2
I20251212 08:53:28.833894 5582 raft_consensus.cc:697] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 2 LEADER]: Becoming Leader. State: Replica: 513456cf96614495a7c46e42b3a33e55, State: Running, Role: LEADER
I20251212 08:53:28.834115 5582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER }
I20251212 08:53:28.834187 5579 sys_catalog.cc:565] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:28.834862 5584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 513456cf96614495a7c46e42b3a33e55. Latest consensus state: current_term: 2 leader_uuid: "513456cf96614495a7c46e42b3a33e55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER } }
I20251212 08:53:28.835005 5584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:28.834918 5583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "513456cf96614495a7c46e42b3a33e55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "513456cf96614495a7c46e42b3a33e55" member_type: VOTER } }
I20251212 08:53:28.835224 5583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:28.836241 5587 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:28.837714 5587 catalog_manager.cc:679] Loaded metadata for table client-testtb [id=17e639d5a7b84efcbd103c28c41dc4e1]
I20251212 08:53:28.838857 5587 tablet_loader.cc:96] loaded metadata for tablet 1e3079590e0446b687d76a5a3d8d520c (table client-testtb [id=17e639d5a7b84efcbd103c28c41dc4e1])
I20251212 08:53:28.839156 5587 tablet_loader.cc:96] loaded metadata for tablet 2971825ac10a42a3b5dcd2a2911b519b (table client-testtb [id=17e639d5a7b84efcbd103c28c41dc4e1])
I20251212 08:53:28.839316 5587 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:28.839682 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:28.840114 5587 catalog_manager.cc:1269] Loaded cluster ID: a4943d5ef5584bda92652fd5ecf97b63
I20251212 08:53:28.840176 5587 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:28.842005 5587 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:28.842765 5587 catalog_manager.cc:6038] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55: Loaded TSK: 0
I20251212 08:53:28.843313 5587 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:28.847890 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:28.850684 5599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.850795 5600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:28.851094 5602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:28.851126 832 server_base.cc:1047] running on GCE node
I20251212 08:53:28.851404 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:28.851459 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:28.851521 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529608851517 us; error 0 us; skew 500 ppm
I20251212 08:53:28.851665 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:28.852919 832 webserver.cc:492] Webserver started at http://127.0.208.1:36191/ using document root <none> and password file <none>
I20251212 08:53:28.853137 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:28.853191 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:28.855007 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:28.856348 5607 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:28.856693 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:28.856792 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "432b7b164a0e4d21b8f76b17a230d1cf"
format_stamp: "Formatted at 2025-12-12 08:53:28 on dist-test-slave-qdpb"
I20251212 08:53:28.856906 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TestBlockScannerHijackingAttempts.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:28.864244 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:28.864785 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:28.867189 5617 ts_tablet_manager.cc:548] Loading tablet metadata (0/2 complete)
I20251212 08:53:28.873359 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:28.873452 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20251212 08:53:28.873569 832 ts_tablet_manager.cc:600] Registering tablets (0/2 complete)
I20251212 08:53:28.876168 5617 tablet_bootstrap.cc:492] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: Bootstrap starting.
I20251212 08:53:28.877800 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:28.877876 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:28.881345 5617 tablet_bootstrap.cc:492] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:28.881847 5617 tablet_bootstrap.cc:492] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: Bootstrap complete.
I20251212 08:53:28.882153 5617 ts_tablet_manager.cc:1403] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20251212 08:53:28.882943 5617 raft_consensus.cc:359] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.883104 5617 raft_consensus.cc:740] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432b7b164a0e4d21b8f76b17a230d1cf, State: Initialized, Role: FOLLOWER
I20251212 08:53:28.883324 5617 consensus_queue.cc:260] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.883502 5617 raft_consensus.cc:399] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:28.883569 5617 raft_consensus.cc:493] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:28.883652 5617 raft_consensus.cc:3060] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Advancing to term 2
I20251212 08:53:28.885808 5617 raft_consensus.cc:515] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.886016 5617 leader_election.cc:304] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 432b7b164a0e4d21b8f76b17a230d1cf; no voters:
I20251212 08:53:28.886567 5617 leader_election.cc:290] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [CANDIDATE]: Term 2 election: Requested vote from peers
I20251212 08:53:28.886734 5643 raft_consensus.cc:2804] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 FOLLOWER]: Leader election won for term 2
I20251212 08:53:28.890509 5643 raft_consensus.cc:697] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 LEADER]: Becoming Leader. State: Replica: 432b7b164a0e4d21b8f76b17a230d1cf, State: Running, Role: LEADER
I20251212 08:53:28.890854 5643 consensus_queue.cc:237] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.892141 5617 ts_tablet_manager.cc:1434] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf: Time spent starting tablet: real 0.010s user 0.002s sys 0.007s
I20251212 08:53:28.892421 5617 tablet_bootstrap.cc:492] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: Bootstrap starting.
I20251212 08:53:28.896684 5617 tablet_bootstrap.cc:492] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:28.897017 5617 tablet_bootstrap.cc:492] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: Bootstrap complete.
I20251212 08:53:28.897264 5617 ts_tablet_manager.cc:1403] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20251212 08:53:28.897943 5617 raft_consensus.cc:359] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.898072 5617 raft_consensus.cc:740] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432b7b164a0e4d21b8f76b17a230d1cf, State: Initialized, Role: FOLLOWER
I20251212 08:53:28.898284 5617 consensus_queue.cc:260] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.898397 5617 raft_consensus.cc:399] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:28.898459 5617 raft_consensus.cc:493] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:28.898550 5617 raft_consensus.cc:3060] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 1 FOLLOWER]: Advancing to term 2
I20251212 08:53:28.901202 5617 raft_consensus.cc:515] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.901464 5617 leader_election.cc:304] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 432b7b164a0e4d21b8f76b17a230d1cf; no voters:
I20251212 08:53:28.901643 5617 leader_election.cc:290] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [CANDIDATE]: Term 2 election: Requested vote from peers
I20251212 08:53:28.901707 5647 raft_consensus.cc:2804] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 FOLLOWER]: Leader election won for term 2
I20251212 08:53:28.901844 5647 raft_consensus.cc:697] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 LEADER]: Becoming Leader. State: Replica: 432b7b164a0e4d21b8f76b17a230d1cf, State: Running, Role: LEADER
I20251212 08:53:28.902093 5647 consensus_queue.cc:237] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } }
I20251212 08:53:28.902410 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:37981
I20251212 08:53:28.902467 5685 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:37981 every 8 connection(s)
I20251212 08:53:28.903185 5617 ts_tablet_manager.cc:1434] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20251212 08:53:28.904888 5687 heartbeater.cc:344] Connected to a master server at 127.0.208.62:38535
I20251212 08:53:28.905210 5687 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:28.905791 5687 heartbeater.cc:507] Master 127.0.208.62:38535 requested a full tablet report, sending...
I20251212 08:53:28.907847 5534 ts_manager.cc:194] Registered new tserver with Master: 432b7b164a0e4d21b8f76b17a230d1cf (127.0.208.1:37981)
I20251212 08:53:28.908351 5534 catalog_manager.cc:5654] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:28.908691 5534 catalog_manager.cc:5654] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "432b7b164a0e4d21b8f76b17a230d1cf" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38381 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:28.908735 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.001199302s
I20251212 08:53:28.913946 5534 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36504
I20251212 08:53:28.915663 5687 heartbeater.cc:499] Master 127.0.208.62:38535 was elected leader, sending a full tablet report...
I20251212 08:53:29.060539 832 client-test.cc:7897] Attempting to extract data from fault-tolerant scan 1c24e22cee38460884af2fdf18e5841f as bad-guy
I20251212 08:53:29.061205 5634 tablet_service.cc:3173] Scan: Not authorized: User bad-guy doesn't own scanner 1c24e22cee38460884af2fdf18e5841f: call sequence id=1, remote={username='bad-guy'} at 127.0.0.1:44364
W20251212 08:53:29.061596 832 client.cc:2235] Scan on tablet server 432b7b164a0e4d21b8f76b17a230d1cf (127.0.208.1:37981) with Scanner { table: client-testtb, tablet: 1e3079590e0446b687d76a5a3d8d520c, projection: (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
), scan_spec: } failed: Remote error: Not authorized: User bad-guy doesn't own scanner 1c24e22cee38460884af2fdf18e5841f
I20251212 08:53:29.062325 5634 tablet_service.cc:3173] Scan: Not authorized: User bad-guy doesn't own scanner 1c24e22cee38460884af2fdf18e5841f: call sequence id=2, remote={username='bad-guy'} at 127.0.0.1:44364
W20251212 08:53:29.062642 5702 client.cc:2042] Couldn't close scanner 1c24e22cee38460884af2fdf18e5841f: Remote error: Not authorized: User bad-guy doesn't own scanner 1c24e22cee38460884af2fdf18e5841f
I20251212 08:53:29.066455 832 client-test.cc:7897] Attempting to extract data from non-fault-tolerant scan e4eff87b83714b86845e4627103018c6 as bad-guy
I20251212 08:53:29.067032 5635 tablet_service.cc:3173] Scan: Not authorized: User bad-guy doesn't own scanner e4eff87b83714b86845e4627103018c6: call sequence id=1, remote={username='bad-guy'} at 127.0.0.1:44364
W20251212 08:53:29.067447 832 client.cc:2235] Scan on tablet server 432b7b164a0e4d21b8f76b17a230d1cf (127.0.208.1:37981) with Scanner { table: client-testtb, tablet: 1e3079590e0446b687d76a5a3d8d520c, projection: (
10:key INT32 NOT NULL,
11:int_val INT32 NOT NULL,
12:string_val STRING NULLABLE,
13:non_null_with_default INT32 NOT NULL,
PRIMARY KEY (key)
), scan_spec: } failed: Remote error: Not authorized: User bad-guy doesn't own scanner e4eff87b83714b86845e4627103018c6
I20251212 08:53:29.068140 5635 tablet_service.cc:3173] Scan: Not authorized: User bad-guy doesn't own scanner e4eff87b83714b86845e4627103018c6: call sequence id=2, remote={username='bad-guy'} at 127.0.0.1:44364
W20251212 08:53:29.068519 5702 client.cc:2042] Couldn't close scanner e4eff87b83714b86845e4627103018c6: Remote error: Not authorized: User bad-guy doesn't own scanner e4eff87b83714b86845e4627103018c6
I20251212 08:53:29.070966 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:29.080489 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:29.080919 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:29.081133 832 raft_consensus.cc:2243] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 LEADER]: Raft consensus shutting down.
I20251212 08:53:29.081327 832 raft_consensus.cc:2272] T 2971825ac10a42a3b5dcd2a2911b519b P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:29.082163 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:29.082271 832 raft_consensus.cc:2243] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 LEADER]: Raft consensus shutting down.
I20251212 08:53:29.082401 832 raft_consensus.cc:2272] T 1e3079590e0446b687d76a5a3d8d520c P 432b7b164a0e4d21b8f76b17a230d1cf [term 2 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:29.096874 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:29.102494 832 master.cc:561] Master@127.0.208.62:38535 shutting down...
I20251212 08:53:29.111400 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 2 LEADER]: Raft consensus shutting down.
I20251212 08:53:29.111675 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 513456cf96614495a7c46e42b3a33e55 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:29.111809 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:29.125886 832 master.cc:583] Master@127.0.208.62:38535 shutdown complete.
[ OK ] ClientTest.TestBlockScannerHijackingAttempts (646 ms)
[ RUN ] ClientTest.FlushTxnSessionsOnCommit
I20251212 08:53:29.135991 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:40141
I20251212 08:53:29.136526 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:29.138756 5707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:29.138751 5708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:29.139189 5710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:29.139335 832 server_base.cc:1047] running on GCE node
I20251212 08:53:29.139601 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:29.139655 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:29.139689 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529609139685 us; error 0 us; skew 500 ppm
I20251212 08:53:29.139829 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:29.141045 832 webserver.cc:492] Webserver started at http://127.0.208.62:38421/ using document root <none> and password file <none>
I20251212 08:53:29.141238 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:29.141304 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:29.141481 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:29.148056 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "4462d1fc74114eefa8bf19c1513b841d"
format_stamp: "Formatted at 2025-12-12 08:53:29 on dist-test-slave-qdpb"
I20251212 08:53:29.150571 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:29.152097 5715 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:29.152462 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:29.152560 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "4462d1fc74114eefa8bf19c1513b841d"
format_stamp: "Formatted at 2025-12-12 08:53:29 on dist-test-slave-qdpb"
I20251212 08:53:29.152673 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:29.177170 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:29.177675 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:29.191890 5776 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:40141 every 8 connection(s)
I20251212 08:53:29.191885 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:40141
I20251212 08:53:29.193192 5777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:29.198231 5777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d: Bootstrap starting.
I20251212 08:53:29.199432 5777 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:29.201052 5777 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d: No bootstrap required, opened a new log
I20251212 08:53:29.201685 5777 raft_consensus.cc:359] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4462d1fc74114eefa8bf19c1513b841d" member_type: VOTER }
I20251212 08:53:29.201818 5777 raft_consensus.cc:385] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:29.201891 5777 raft_consensus.cc:740] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4462d1fc74114eefa8bf19c1513b841d, State: Initialized, Role: FOLLOWER
I20251212 08:53:29.202082 5777 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4462d1fc74114eefa8bf19c1513b841d" member_type: VOTER }
I20251212 08:53:29.202190 5777 raft_consensus.cc:399] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:29.202253 5777 raft_consensus.cc:493] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:29.202332 5777 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:29.204334 5777 raft_consensus.cc:515] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4462d1fc74114eefa8bf19c1513b841d" member_type: VOTER }
I20251212 08:53:29.204517 5777 leader_election.cc:304] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4462d1fc74114eefa8bf19c1513b841d; no voters:
I20251212 08:53:29.204931 5777 leader_election.cc:290] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:29.205082 5780 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:29.205531 5780 raft_consensus.cc:697] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 1 LEADER]: Becoming Leader. State: Replica: 4462d1fc74114eefa8bf19c1513b841d, State: Running, Role: LEADER
I20251212 08:53:29.205754 5777 sys_catalog.cc:565] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:29.205770 5780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4462d1fc74114eefa8bf19c1513b841d" member_type: VOTER }
I20251212 08:53:29.206754 5781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4462d1fc74114eefa8bf19c1513b841d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4462d1fc74114eefa8bf19c1513b841d" member_type: VOTER } }
I20251212 08:53:29.206908 5782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4462d1fc74114eefa8bf19c1513b841d. Latest consensus state: current_term: 1 leader_uuid: "4462d1fc74114eefa8bf19c1513b841d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4462d1fc74114eefa8bf19c1513b841d" member_type: VOTER } }
I20251212 08:53:29.207011 5781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:29.207046 5782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:29.211474 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20251212 08:53:29.212234 5796 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251212 08:53:29.212358 5796 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251212 08:53:29.212440 5786 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:29.213420 5786 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:29.216373 5786 catalog_manager.cc:1357] Generated new cluster ID: 0f685dfb81a5406dba3bd3fcb5d4452f
I20251212 08:53:29.216471 5786 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:29.226897 5786 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:29.227701 5786 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:29.233525 5786 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d: Generated new TSK 0
I20251212 08:53:29.233742 5786 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:29.244498 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:29.247452 5799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:29.247453 5798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:29.247856 5801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:29.248134 832 server_base.cc:1047] running on GCE node
I20251212 08:53:29.248396 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:29.248450 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:29.248478 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529609248471 us; error 0 us; skew 500 ppm
I20251212 08:53:29.248584 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:29.249819 832 webserver.cc:492] Webserver started at http://127.0.208.1:34435/ using document root <none> and password file <none>
I20251212 08:53:29.250007 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:29.250065 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:29.250159 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:29.250720 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "26beb27e74554179af24a4357a9d5580"
format_stamp: "Formatted at 2025-12-12 08:53:29 on dist-test-slave-qdpb"
I20251212 08:53:29.253216 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:29.254648 5806 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:29.254966 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:29.255053 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "26beb27e74554179af24a4357a9d5580"
format_stamp: "Formatted at 2025-12-12 08:53:29 on dist-test-slave-qdpb"
I20251212 08:53:29.255151 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.FlushTxnSessionsOnCommit.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:29.262310 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:29.262825 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:29.264364 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:29.264427 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:29.264506 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:29.264554 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:29.282292 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:36119
I20251212 08:53:29.282325 5876 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:36119 every 8 connection(s)
I20251212 08:53:29.284062 5877 heartbeater.cc:344] Connected to a master server at 127.0.208.62:40141
I20251212 08:53:29.284197 5877 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:29.284559 5877 heartbeater.cc:507] Master 127.0.208.62:40141 requested a full tablet report, sending...
I20251212 08:53:29.285411 5732 ts_manager.cc:194] Registered new tserver with Master: 26beb27e74554179af24a4357a9d5580 (127.0.208.1:36119)
I20251212 08:53:29.285666 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002362862s
I20251212 08:53:29.286504 5732 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37528
I20251212 08:53:29.296818 5732 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37552:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:29.297837 5877 heartbeater.cc:499] Master 127.0.208.62:40141 was elected leader, sending a full tablet report...
I20251212 08:53:29.314586 5839 tablet_service.cc:1505] Processing CreateTablet for tablet bc04c4bb32394908a4a50f5424c01eb9 (DEFAULT_TABLE table=client-testtb [id=eb33cd4daf0d4f3493c405bff170394e]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:29.314586 5837 tablet_service.cc:1505] Processing CreateTablet for tablet 06f56a926f3840768178c8fbc409abb2 (DEFAULT_TABLE table=client-testtb [id=eb33cd4daf0d4f3493c405bff170394e]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:29.315160 5839 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc04c4bb32394908a4a50f5424c01eb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:29.315357 5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 06f56a926f3840768178c8fbc409abb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:29.322762 5887 tablet_bootstrap.cc:492] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580: Bootstrap starting.
I20251212 08:53:29.323926 5887 tablet_bootstrap.cc:654] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:29.325356 5887 tablet_bootstrap.cc:492] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580: No bootstrap required, opened a new log
I20251212 08:53:29.325479 5887 ts_tablet_manager.cc:1403] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:53:29.326035 5887 raft_consensus.cc:359] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.326165 5887 raft_consensus.cc:385] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:29.326226 5887 raft_consensus.cc:740] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26beb27e74554179af24a4357a9d5580, State: Initialized, Role: FOLLOWER
I20251212 08:53:29.326406 5887 consensus_queue.cc:260] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.326510 5887 raft_consensus.cc:399] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:29.326591 5887 raft_consensus.cc:493] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:29.326666 5887 raft_consensus.cc:3060] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:29.329000 5887 raft_consensus.cc:515] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.329171 5887 leader_election.cc:304] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26beb27e74554179af24a4357a9d5580; no voters:
I20251212 08:53:29.329519 5887 leader_election.cc:290] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:29.329654 5889 raft_consensus.cc:2804] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:29.330101 5889 raft_consensus.cc:697] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 1 LEADER]: Becoming Leader. State: Replica: 26beb27e74554179af24a4357a9d5580, State: Running, Role: LEADER
I20251212 08:53:29.330150 5887 ts_tablet_manager.cc:1434] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20251212 08:53:29.330387 5887 tablet_bootstrap.cc:492] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580: Bootstrap starting.
I20251212 08:53:29.330346 5889 consensus_queue.cc:237] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.331704 5887 tablet_bootstrap.cc:654] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:29.332475 5732 catalog_manager.cc:5654] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26beb27e74554179af24a4357a9d5580 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "26beb27e74554179af24a4357a9d5580" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:29.333519 5887 tablet_bootstrap.cc:492] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580: No bootstrap required, opened a new log
I20251212 08:53:29.333647 5887 ts_tablet_manager.cc:1403] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:53:29.334333 5887 raft_consensus.cc:359] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.334488 5887 raft_consensus.cc:385] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:29.334555 5887 raft_consensus.cc:740] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26beb27e74554179af24a4357a9d5580, State: Initialized, Role: FOLLOWER
I20251212 08:53:29.334739 5887 consensus_queue.cc:260] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.334915 5887 raft_consensus.cc:399] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:29.334985 5887 raft_consensus.cc:493] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:29.335047 5887 raft_consensus.cc:3060] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:29.337503 5887 raft_consensus.cc:515] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.337728 5887 leader_election.cc:304] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26beb27e74554179af24a4357a9d5580; no voters:
I20251212 08:53:29.337903 5887 leader_election.cc:290] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:29.338011 5889 raft_consensus.cc:2804] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:29.338222 5889 raft_consensus.cc:697] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 1 LEADER]: Becoming Leader. State: Replica: 26beb27e74554179af24a4357a9d5580, State: Running, Role: LEADER
I20251212 08:53:29.338408 5887 ts_tablet_manager.cc:1434] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580: Time spent starting tablet: real 0.005s user 0.000s sys 0.003s
I20251212 08:53:29.338677 5889 consensus_queue.cc:237] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.341756 5732 catalog_manager.cc:5654] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26beb27e74554179af24a4357a9d5580 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "26beb27e74554179af24a4357a9d5580" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:29.366353 5732 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37560:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
I20251212 08:53:29.373389 5837 tablet_service.cc:1505] Processing CreateTablet for tablet 93402333a0f34b2ca48c25294fa3f68f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=b4461fb6bb7a4e47a361af8c83acf96e]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20251212 08:53:29.373876 5837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93402333a0f34b2ca48c25294fa3f68f. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:29.379259 5887 tablet_bootstrap.cc:492] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: Bootstrap starting.
I20251212 08:53:29.380573 5887 tablet_bootstrap.cc:654] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:29.382066 5887 tablet_bootstrap.cc:492] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: No bootstrap required, opened a new log
I20251212 08:53:29.382184 5887 ts_tablet_manager.cc:1403] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:29.382949 5887 raft_consensus.cc:359] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.383080 5887 raft_consensus.cc:385] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:29.383116 5887 raft_consensus.cc:740] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26beb27e74554179af24a4357a9d5580, State: Initialized, Role: FOLLOWER
I20251212 08:53:29.383329 5887 consensus_queue.cc:260] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.383476 5887 raft_consensus.cc:399] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:29.383538 5887 raft_consensus.cc:493] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:29.383595 5887 raft_consensus.cc:3060] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:29.385560 5887 raft_consensus.cc:515] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.385737 5887 leader_election.cc:304] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26beb27e74554179af24a4357a9d5580; no voters:
I20251212 08:53:29.385882 5887 leader_election.cc:290] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:29.385964 5891 raft_consensus.cc:2804] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:29.386092 5891 raft_consensus.cc:697] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 1 LEADER]: Becoming Leader. State: Replica: 26beb27e74554179af24a4357a9d5580, State: Running, Role: LEADER
I20251212 08:53:29.386351 5891 consensus_queue.cc:237] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } }
I20251212 08:53:29.386525 5887 ts_tablet_manager.cc:1434] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20251212 08:53:29.387627 5889 tablet_replica.cc:442] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26beb27e74554179af24a4357a9d5580" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } } }
I20251212 08:53:29.387818 5889 tablet_replica.cc:445] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: This TxnStatusTablet replica's current role is: LEADER
I20251212 08:53:29.388140 5890 tablet_replica.cc:442] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: TxnStatusTablet state changed. Reason: New leader 26beb27e74554179af24a4357a9d5580. Latest consensus state: current_term: 1 leader_uuid: "26beb27e74554179af24a4357a9d5580" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } } }
I20251212 08:53:29.388269 5890 tablet_replica.cc:445] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: This TxnStatusTablet replica's current role is: LEADER
I20251212 08:53:29.388672 5903 txn_status_manager.cc:874] Waiting until node catch up with all replicated operations in previous term...
I20251212 08:53:29.388892 5903 txn_status_manager.cc:930] Loading transaction status metadata into memory...
I20251212 08:53:29.389192 5732 catalog_manager.cc:5654] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26beb27e74554179af24a4357a9d5580 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "26beb27e74554179af24a4357a9d5580" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26beb27e74554179af24a4357a9d5580" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36119 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:29.986544 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:30.007048 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:30.007511 832 tablet_replica.cc:333] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580: stopping tablet replica
I20251212 08:53:30.007772 832 raft_consensus.cc:2243] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:30.007998 832 raft_consensus.cc:2272] T 93402333a0f34b2ca48c25294fa3f68f P 26beb27e74554179af24a4357a9d5580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:30.009161 832 tablet_replica.cc:333] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580: stopping tablet replica
I20251212 08:53:30.009312 832 raft_consensus.cc:2243] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:30.009482 832 raft_consensus.cc:2272] T bc04c4bb32394908a4a50f5424c01eb9 P 26beb27e74554179af24a4357a9d5580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:30.010349 832 tablet_replica.cc:333] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580: stopping tablet replica
I20251212 08:53:30.010480 832 raft_consensus.cc:2243] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:30.010618 832 raft_consensus.cc:2272] T 06f56a926f3840768178c8fbc409abb2 P 26beb27e74554179af24a4357a9d5580 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:30.016649 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:30.022996 832 master.cc:561] Master@127.0.208.62:40141 shutting down...
I20251212 08:53:30.036144 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:30.036463 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:30.036605 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 4462d1fc74114eefa8bf19c1513b841d: stopping tablet replica
I20251212 08:53:30.051801 832 master.cc:583] Master@127.0.208.62:40141 shutdown complete.
[ OK ] ClientTest.FlushTxnSessionsOnCommit (927 ms)
[ RUN ] ClientTest.TxnCorruptedToken
I20251212 08:53:30.064095 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:41027
I20251212 08:53:30.064744 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:30.067374 5914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:30.068084 5915 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:30.069427 5917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:30.069689 832 server_base.cc:1047] running on GCE node
I20251212 08:53:30.069964 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:30.070019 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:30.070055 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529610070050 us; error 0 us; skew 500 ppm
I20251212 08:53:30.070194 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:30.071455 832 webserver.cc:492] Webserver started at http://127.0.208.62:40037/ using document root <none> and password file <none>
I20251212 08:53:30.071651 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:30.071717 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:30.071838 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:30.072512 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "475d6da5e29f454898cff8483e93b15d"
format_stamp: "Formatted at 2025-12-12 08:53:30 on dist-test-slave-qdpb"
I20251212 08:53:30.075089 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251212 08:53:30.076884 5922 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:30.077288 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:30.077409 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "475d6da5e29f454898cff8483e93b15d"
format_stamp: "Formatted at 2025-12-12 08:53:30 on dist-test-slave-qdpb"
I20251212 08:53:30.077528 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:30.096783 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:30.097482 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:30.112937 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:41027
I20251212 08:53:30.112968 5983 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:41027 every 8 connection(s)
I20251212 08:53:30.114444 5984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.119345 5984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d: Bootstrap starting.
I20251212 08:53:30.120627 5984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.122133 5984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d: No bootstrap required, opened a new log
I20251212 08:53:30.122735 5984 raft_consensus.cc:359] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475d6da5e29f454898cff8483e93b15d" member_type: VOTER }
I20251212 08:53:30.122875 5984 raft_consensus.cc:385] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.122941 5984 raft_consensus.cc:740] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475d6da5e29f454898cff8483e93b15d, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.123152 5984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475d6da5e29f454898cff8483e93b15d" member_type: VOTER }
I20251212 08:53:30.123288 5984 raft_consensus.cc:399] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.123351 5984 raft_consensus.cc:493] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.123426 5984 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.125571 5984 raft_consensus.cc:515] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475d6da5e29f454898cff8483e93b15d" member_type: VOTER }
I20251212 08:53:30.125762 5984 leader_election.cc:304] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 475d6da5e29f454898cff8483e93b15d; no voters:
I20251212 08:53:30.126226 5984 leader_election.cc:290] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.126340 5987 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.127562 5984 sys_catalog.cc:565] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:30.127928 5987 raft_consensus.cc:697] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 1 LEADER]: Becoming Leader. State: Replica: 475d6da5e29f454898cff8483e93b15d, State: Running, Role: LEADER
I20251212 08:53:30.128196 5987 consensus_queue.cc:237] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475d6da5e29f454898cff8483e93b15d" member_type: VOTER }
I20251212 08:53:30.133375 5988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "475d6da5e29f454898cff8483e93b15d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475d6da5e29f454898cff8483e93b15d" member_type: VOTER } }
I20251212 08:53:30.133675 5988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:30.134171 5989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 475d6da5e29f454898cff8483e93b15d. Latest consensus state: current_term: 1 leader_uuid: "475d6da5e29f454898cff8483e93b15d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "475d6da5e29f454898cff8483e93b15d" member_type: VOTER } }
I20251212 08:53:30.134356 5989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:30.141546 5994 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:30.143330 5994 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:30.144297 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:30.146896 5994 catalog_manager.cc:1357] Generated new cluster ID: 0d1fe7ee69c745528727f808b7af88d7
I20251212 08:53:30.146996 5994 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:30.157904 5994 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:30.158852 5994 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:30.174095 5994 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d: Generated new TSK 0
I20251212 08:53:30.174343 5994 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:30.177192 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:30.180683 6007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:30.180686 6006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:30.181026 6009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:30.181424 832 server_base.cc:1047] running on GCE node
I20251212 08:53:30.181697 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:30.181785 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:30.181829 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529610181825 us; error 0 us; skew 500 ppm
I20251212 08:53:30.181967 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:30.183184 832 webserver.cc:492] Webserver started at http://127.0.208.1:41737/ using document root <none> and password file <none>
I20251212 08:53:30.183393 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:30.183465 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:30.183579 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:30.184235 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "0bef548ef95c496faafbb19c9d3b01bd"
format_stamp: "Formatted at 2025-12-12 08:53:30 on dist-test-slave-qdpb"
I20251212 08:53:30.186779 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:30.188462 6014 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:30.188886 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:53:30.188997 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "0bef548ef95c496faafbb19c9d3b01bd"
format_stamp: "Formatted at 2025-12-12 08:53:30 on dist-test-slave-qdpb"
I20251212 08:53:30.189100 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTest.TxnCorruptedToken.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:30.209185 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:30.209756 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:30.225162 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:30.225255 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:30.225351 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:30.225399 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:30.268687 6084 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:40713 every 8 connection(s)
I20251212 08:53:30.268687 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:40713
I20251212 08:53:30.271379 6085 heartbeater.cc:344] Connected to a master server at 127.0.208.62:41027
I20251212 08:53:30.271574 6085 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:30.272111 6085 heartbeater.cc:507] Master 127.0.208.62:41027 requested a full tablet report, sending...
I20251212 08:53:30.273160 5939 ts_manager.cc:194] Registered new tserver with Master: 0bef548ef95c496faafbb19c9d3b01bd (127.0.208.1:40713)
I20251212 08:53:30.275233 5939 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54444
I20251212 08:53:30.278261 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.000124155s
I20251212 08:53:30.287647 6085 heartbeater.cc:499] Master 127.0.208.62:41027 was elected leader, sending a full tablet report...
I20251212 08:53:30.302455 5939 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54466:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:30.319130 6048 tablet_service.cc:1505] Processing CreateTablet for tablet ae3576bd73e546e29ec6f619dd8d4339 (DEFAULT_TABLE table=client-testtb [id=3e911db7b59844ac88b99e7663b85a00]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:30.319284 6046 tablet_service.cc:1505] Processing CreateTablet for tablet 19c2b4c97b414f43a76217d96179087c (DEFAULT_TABLE table=client-testtb [id=3e911db7b59844ac88b99e7663b85a00]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:30.319639 6048 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ae3576bd73e546e29ec6f619dd8d4339. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.319986 6046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19c2b4c97b414f43a76217d96179087c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.325673 6095 tablet_bootstrap.cc:492] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd: Bootstrap starting.
I20251212 08:53:30.327389 6095 tablet_bootstrap.cc:654] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.329563 6095 tablet_bootstrap.cc:492] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd: No bootstrap required, opened a new log
I20251212 08:53:30.329699 6095 ts_tablet_manager.cc:1403] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20251212 08:53:30.330870 6095 raft_consensus.cc:359] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } }
I20251212 08:53:30.331041 6095 raft_consensus.cc:385] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.331079 6095 raft_consensus.cc:740] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bef548ef95c496faafbb19c9d3b01bd, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.331260 6095 consensus_queue.cc:260] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } }
I20251212 08:53:30.331430 6095 raft_consensus.cc:399] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.331502 6095 raft_consensus.cc:493] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.331611 6095 raft_consensus.cc:3060] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.333801 6095 raft_consensus.cc:515] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } }
I20251212 08:53:30.333994 6095 leader_election.cc:304] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bef548ef95c496faafbb19c9d3b01bd; no voters:
I20251212 08:53:30.334450 6095 leader_election.cc:290] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.334630 6097 raft_consensus.cc:2804] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.335142 6095 ts_tablet_manager.cc:1434] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20251212 08:53:30.335248 6097 raft_consensus.cc:697] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 1 LEADER]: Becoming Leader. State: Replica: 0bef548ef95c496faafbb19c9d3b01bd, State: Running, Role: LEADER
I20251212 08:53:30.335415 6095 tablet_bootstrap.cc:492] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd: Bootstrap starting.
I20251212 08:53:30.335484 6097 consensus_queue.cc:237] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } }
I20251212 08:53:30.337210 6095 tablet_bootstrap.cc:654] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.338551 5938 catalog_manager.cc:5654] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bef548ef95c496faafbb19c9d3b01bd (127.0.208.1). New cstate: current_term: 1 leader_uuid: "0bef548ef95c496faafbb19c9d3b01bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:30.343514 6095 tablet_bootstrap.cc:492] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd: No bootstrap required, opened a new log
I20251212 08:53:30.343669 6095 ts_tablet_manager.cc:1403] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20251212 08:53:30.344393 6095 raft_consensus.cc:359] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } }
I20251212 08:53:30.344575 6095 raft_consensus.cc:385] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.344640 6095 raft_consensus.cc:740] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bef548ef95c496faafbb19c9d3b01bd, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.344872 6095 consensus_queue.cc:260] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } }
I20251212 08:53:30.345016 6095 raft_consensus.cc:399] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.345083 6095 raft_consensus.cc:493] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.345167 6095 raft_consensus.cc:3060] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.347321 6095 raft_consensus.cc:515] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } }
I20251212 08:53:30.347548 6095 leader_election.cc:304] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0bef548ef95c496faafbb19c9d3b01bd; no voters:
I20251212 08:53:30.347746 6095 leader_election.cc:290] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.347852 6097 raft_consensus.cc:2804] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.348019 6097 raft_consensus.cc:697] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 1 LEADER]: Becoming Leader. State: Replica: 0bef548ef95c496faafbb19c9d3b01bd, State: Running, Role: LEADER
I20251212 08:53:30.348171 6095 ts_tablet_manager.cc:1434] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:30.348258 6097 consensus_queue.cc:237] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } }
I20251212 08:53:30.350677 5938 catalog_manager.cc:5654] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd reported cstate change: term changed from 0 to 1, leader changed from <none> to 0bef548ef95c496faafbb19c9d3b01bd (127.0.208.1). New cstate: current_term: 1 leader_uuid: "0bef548ef95c496faafbb19c9d3b01bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0bef548ef95c496faafbb19c9d3b01bd" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40713 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:30.356487 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:30.367919 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:30.368367 832 tablet_replica.cc:333] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd: stopping tablet replica
I20251212 08:53:30.368599 832 raft_consensus.cc:2243] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:30.368798 832 raft_consensus.cc:2272] T ae3576bd73e546e29ec6f619dd8d4339 P 0bef548ef95c496faafbb19c9d3b01bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:30.369736 832 tablet_replica.cc:333] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd: stopping tablet replica
I20251212 08:53:30.369885 832 raft_consensus.cc:2243] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:30.370045 832 raft_consensus.cc:2272] T 19c2b4c97b414f43a76217d96179087c P 0bef548ef95c496faafbb19c9d3b01bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:30.384948 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:30.389364 832 master.cc:561] Master@127.0.208.62:41027 shutting down...
I20251212 08:53:30.399490 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:30.399978 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:30.400417 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 475d6da5e29f454898cff8483e93b15d: stopping tablet replica
I20251212 08:53:30.406749 832 master.cc:583] Master@127.0.208.62:41027 shutdown complete.
[ OK ] ClientTest.TxnCorruptedToken (352 ms)
[----------] 20 tests from ClientTest (53175 ms total)
[----------] 1 test from TableKeyRangeTest
[ RUN ] TableKeyRangeTest.TestGetTableKeyRange
I20251212 08:53:30.416947 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:32773
I20251212 08:53:30.417497 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:30.420003 6104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:30.420156 6107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:30.420131 6105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:30.420699 832 server_base.cc:1047] running on GCE node
I20251212 08:53:30.421043 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:30.421104 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:30.421139 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529610421135 us; error 0 us; skew 500 ppm
I20251212 08:53:30.421286 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:30.422562 832 webserver.cc:492] Webserver started at http://127.0.208.62:33527/ using document root <none> and password file <none>
I20251212 08:53:30.422789 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:30.422855 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:30.422959 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:30.423620 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "278fbf43d3ed4e08ab6210169db2a0da"
format_stamp: "Formatted at 2025-12-12 08:53:30 on dist-test-slave-qdpb"
I20251212 08:53:30.426467 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20251212 08:53:30.428125 6112 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:30.428565 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:53:30.428663 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "278fbf43d3ed4e08ab6210169db2a0da"
format_stamp: "Formatted at 2025-12-12 08:53:30 on dist-test-slave-qdpb"
I20251212 08:53:30.428778 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:30.440270 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:30.440863 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:30.455317 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:32773
I20251212 08:53:30.455354 6173 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:32773 every 8 connection(s)
I20251212 08:53:30.456787 6174 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.461630 6174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da: Bootstrap starting.
I20251212 08:53:30.462985 6174 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.464742 6174 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da: No bootstrap required, opened a new log
I20251212 08:53:30.465340 6174 raft_consensus.cc:359] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278fbf43d3ed4e08ab6210169db2a0da" member_type: VOTER }
I20251212 08:53:30.465471 6174 raft_consensus.cc:385] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.465518 6174 raft_consensus.cc:740] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 278fbf43d3ed4e08ab6210169db2a0da, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.465695 6174 consensus_queue.cc:260] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278fbf43d3ed4e08ab6210169db2a0da" member_type: VOTER }
I20251212 08:53:30.465816 6174 raft_consensus.cc:399] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.465858 6174 raft_consensus.cc:493] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.465914 6174 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.467926 6174 raft_consensus.cc:515] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278fbf43d3ed4e08ab6210169db2a0da" member_type: VOTER }
I20251212 08:53:30.468120 6174 leader_election.cc:304] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 278fbf43d3ed4e08ab6210169db2a0da; no voters:
I20251212 08:53:30.468580 6174 leader_election.cc:290] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.468724 6177 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.469204 6177 raft_consensus.cc:697] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 1 LEADER]: Becoming Leader. State: Replica: 278fbf43d3ed4e08ab6210169db2a0da, State: Running, Role: LEADER
I20251212 08:53:30.469439 6177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278fbf43d3ed4e08ab6210169db2a0da" member_type: VOTER }
I20251212 08:53:30.469475 6174 sys_catalog.cc:565] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:30.470372 6179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [sys.catalog]: SysCatalogTable state changed. Reason: New leader 278fbf43d3ed4e08ab6210169db2a0da. Latest consensus state: current_term: 1 leader_uuid: "278fbf43d3ed4e08ab6210169db2a0da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278fbf43d3ed4e08ab6210169db2a0da" member_type: VOTER } }
I20251212 08:53:30.470530 6179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:30.470329 6178 sys_catalog.cc:455] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "278fbf43d3ed4e08ab6210169db2a0da" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "278fbf43d3ed4e08ab6210169db2a0da" member_type: VOTER } }
I20251212 08:53:30.470659 6178 sys_catalog.cc:458] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:30.471583 6182 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:30.473003 6182 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:30.474437 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:30.476003 6182 catalog_manager.cc:1357] Generated new cluster ID: c8db35efabba4893856a0ab8e257d7a7
I20251212 08:53:30.476099 6182 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:30.515324 6182 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:30.516152 6182 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:30.524552 6182 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da: Generated new TSK 0
I20251212 08:53:30.524838 6182 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:30.539892 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:30.542779 6195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:30.542840 6196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:30.543839 6198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:30.544078 832 server_base.cc:1047] running on GCE node
I20251212 08:53:30.545545 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:30.545632 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:30.545680 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529610545676 us; error 0 us; skew 500 ppm
I20251212 08:53:30.545845 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:30.547626 832 webserver.cc:492] Webserver started at http://127.0.208.1:45383/ using document root <none> and password file <none>
I20251212 08:53:30.547864 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:30.547940 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:30.548067 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:30.548786 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "8c8128523e364ce89100e9b6cfdb065f"
format_stamp: "Formatted at 2025-12-12 08:53:30 on dist-test-slave-qdpb"
I20251212 08:53:30.551434 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:30.553164 6203 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:30.553484 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:30.553607 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8c8128523e364ce89100e9b6cfdb065f"
format_stamp: "Formatted at 2025-12-12 08:53:30 on dist-test-slave-qdpb"
I20251212 08:53:30.553714 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.TableKeyRangeTest.TestGetTableKeyRange.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:30.572151 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:30.572786 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:30.574928 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:30.575043 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:30.575129 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:30.575163 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:30.613696 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:36515
I20251212 08:53:30.613696 6273 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:36515 every 8 connection(s)
I20251212 08:53:30.615849 6274 heartbeater.cc:344] Connected to a master server at 127.0.208.62:32773
I20251212 08:53:30.616024 6274 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:30.616438 6274 heartbeater.cc:507] Master 127.0.208.62:32773 requested a full tablet report, sending...
I20251212 08:53:30.617542 6129 ts_manager.cc:194] Registered new tserver with Master: 8c8128523e364ce89100e9b6cfdb065f (127.0.208.1:36515)
I20251212 08:53:30.618216 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.003472034s
I20251212 08:53:30.618646 6129 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58324
I20251212 08:53:30.629885 6274 heartbeater.cc:499] Master 127.0.208.62:32773 was elected leader, sending a full tablet report...
I20251212 08:53:30.629885 6129 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58338:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:30.645150 6234 tablet_service.cc:1505] Processing CreateTablet for tablet 15129d768dc0498aaeb3ae7bc0207f9e (DEFAULT_TABLE table=client-testtb [id=b2d97269fe42409986fd1273d15fd3d2]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:53:30.645215 6233 tablet_service.cc:1505] Processing CreateTablet for tablet 03b5fd7917574323bd0781409c5cf7c0 (DEFAULT_TABLE table=client-testtb [id=b2d97269fe42409986fd1273d15fd3d2]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:53:30.645602 6234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15129d768dc0498aaeb3ae7bc0207f9e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.645840 6233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 03b5fd7917574323bd0781409c5cf7c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.651870 6284 tablet_bootstrap.cc:492] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f: Bootstrap starting.
I20251212 08:53:30.653198 6284 tablet_bootstrap.cc:654] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.654930 6284 tablet_bootstrap.cc:492] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f: No bootstrap required, opened a new log
I20251212 08:53:30.655079 6284 ts_tablet_manager.cc:1403] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:30.655670 6284 raft_consensus.cc:359] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.655814 6284 raft_consensus.cc:385] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.655881 6284 raft_consensus.cc:740] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.656082 6284 consensus_queue.cc:260] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.656198 6284 raft_consensus.cc:399] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.656260 6284 raft_consensus.cc:493] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.656370 6284 raft_consensus.cc:3060] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.658354 6284 raft_consensus.cc:515] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.658560 6284 leader_election.cc:304] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8128523e364ce89100e9b6cfdb065f; no voters:
I20251212 08:53:30.658982 6284 leader_election.cc:290] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.659158 6286 raft_consensus.cc:2804] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.659658 6284 ts_tablet_manager.cc:1434] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:53:30.659929 6284 tablet_bootstrap.cc:492] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f: Bootstrap starting.
I20251212 08:53:30.661115 6284 tablet_bootstrap.cc:654] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.662110 6286 raft_consensus.cc:697] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Becoming Leader. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Running, Role: LEADER
I20251212 08:53:30.662396 6286 consensus_queue.cc:237] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.662765 6284 tablet_bootstrap.cc:492] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f: No bootstrap required, opened a new log
I20251212 08:53:30.662890 6284 ts_tablet_manager.cc:1403] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:30.663659 6284 raft_consensus.cc:359] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.663792 6284 raft_consensus.cc:385] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.663954 6284 raft_consensus.cc:740] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.664263 6284 consensus_queue.cc:260] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.664559 6284 raft_consensus.cc:399] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.664669 6284 raft_consensus.cc:493] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.664793 6284 raft_consensus.cc:3060] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.665032 6129 catalog_manager.cc:5654] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8128523e364ce89100e9b6cfdb065f (127.0.208.1). New cstate: current_term: 1 leader_uuid: "8c8128523e364ce89100e9b6cfdb065f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:30.667716 6284 raft_consensus.cc:515] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.667979 6284 leader_election.cc:304] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8128523e364ce89100e9b6cfdb065f; no voters:
I20251212 08:53:30.668164 6284 leader_election.cc:290] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.668326 6288 raft_consensus.cc:2804] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.668632 6288 raft_consensus.cc:697] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Becoming Leader. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Running, Role: LEADER
I20251212 08:53:30.668717 6284 ts_tablet_manager.cc:1434] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20251212 08:53:30.668948 6288 consensus_queue.cc:237] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.671422 6129 catalog_manager.cc:5654] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8128523e364ce89100e9b6cfdb065f (127.0.208.1). New cstate: current_term: 1 leader_uuid: "8c8128523e364ce89100e9b6cfdb065f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:30.682132 6129 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58338:
name: "client-testrange"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\006\001\000\000\000\000\000\007\001\000\372\000\000\000\006\001\000\372\000\000\000\007\001\000\364\001\000\000\006\001\000\364\001\000\000\007\001\000\356\002\000\000\006\001\000\356\002\000\000\007\001\000\350\003\000\000""\006\001\000\000\000\000\000\007\001\000\372\000\000\000\006\001\000\372\000\000\000\007\001\000\364\001\000\000\006\001\000\364\001\000\000\007\001\000\356\002\000\000\006\001\000\356\002\000\000\007\001\000\350\003\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:53:30.690680 6234 tablet_service.cc:1505] Processing CreateTablet for tablet b75dae41f671452399b44e584630c12e (DEFAULT_TABLE table=client-testrange [id=db93491a0b5f46bb999f0e7331833afa]), partition=RANGE (key) PARTITION 0 <= VALUES < 250
I20251212 08:53:30.690861 6233 tablet_service.cc:1505] Processing CreateTablet for tablet 0904da7de5a44466a482b2af5c21c1e3 (DEFAULT_TABLE table=client-testrange [id=db93491a0b5f46bb999f0e7331833afa]), partition=RANGE (key) PARTITION 250 <= VALUES < 500
I20251212 08:53:30.691254 6234 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b75dae41f671452399b44e584630c12e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.691283 6232 tablet_service.cc:1505] Processing CreateTablet for tablet 1c2156c9f5a340ada54c8d574e85df3e (DEFAULT_TABLE table=client-testrange [id=db93491a0b5f46bb999f0e7331833afa]), partition=RANGE (key) PARTITION 500 <= VALUES < 750
I20251212 08:53:30.691455 6233 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0904da7de5a44466a482b2af5c21c1e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.691649 6232 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1c2156c9f5a340ada54c8d574e85df3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.692181 6118 maintenance_manager.cc:419] P 278fbf43d3ed4e08ab6210169db2a0da: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=1.000000
I20251212 08:53:30.695546 6231 tablet_service.cc:1505] Processing CreateTablet for tablet f35c201cd6e743c5968e97d104a47719 (DEFAULT_TABLE table=client-testrange [id=db93491a0b5f46bb999f0e7331833afa]), partition=RANGE (key) PARTITION 750 <= VALUES < 1000
I20251212 08:53:30.695983 6231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f35c201cd6e743c5968e97d104a47719. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:30.698143 6284 tablet_bootstrap.cc:492] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f: Bootstrap starting.
I20251212 08:53:30.699718 6284 tablet_bootstrap.cc:654] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.709400 6284 tablet_bootstrap.cc:492] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f: No bootstrap required, opened a new log
I20251212 08:53:30.709547 6284 ts_tablet_manager.cc:1403] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.007s
I20251212 08:53:30.708992 6117 maintenance_manager.cc:643] P 278fbf43d3ed4e08ab6210169db2a0da: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.016s user 0.011s sys 0.004s Metrics: {"bytes_written":7932,"cfile_init":1,"dirs.queue_time_us":48,"dirs.run_cpu_time_us":178,"dirs.run_wall_time_us":902,"drs_written":1,"lbm_read_time_us":59,"lbm_reads_lt_1ms":4,"lbm_write_time_us":615,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":11}
I20251212 08:53:30.710379 6118 maintenance_manager.cc:419] P 278fbf43d3ed4e08ab6210169db2a0da: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 581 bytes on disk
I20251212 08:53:30.710309 6284 raft_consensus.cc:359] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.710534 6284 raft_consensus.cc:385] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.710599 6284 raft_consensus.cc:740] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.710806 6284 consensus_queue.cc:260] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.710922 6284 raft_consensus.cc:399] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.710983 6284 raft_consensus.cc:493] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.711056 6284 raft_consensus.cc:3060] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.711265 6117 maintenance_manager.cc:643] P 278fbf43d3ed4e08ab6210169db2a0da: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":92,"lbm_reads_lt_1ms":4}
I20251212 08:53:30.713342 6284 raft_consensus.cc:515] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.713546 6284 leader_election.cc:304] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8128523e364ce89100e9b6cfdb065f; no voters:
I20251212 08:53:30.713704 6284 leader_election.cc:290] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.713814 6288 raft_consensus.cc:2804] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.714017 6288 raft_consensus.cc:697] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Becoming Leader. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Running, Role: LEADER
I20251212 08:53:30.714124 6284 ts_tablet_manager.cc:1434] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20251212 08:53:30.714313 6288 consensus_queue.cc:237] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.714393 6284 tablet_bootstrap.cc:492] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f: Bootstrap starting.
I20251212 08:53:30.715929 6284 tablet_bootstrap.cc:654] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.716552 6128 catalog_manager.cc:5654] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8128523e364ce89100e9b6cfdb065f (127.0.208.1). New cstate: current_term: 1 leader_uuid: "8c8128523e364ce89100e9b6cfdb065f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:30.717512 6284 tablet_bootstrap.cc:492] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f: No bootstrap required, opened a new log
I20251212 08:53:30.717664 6284 ts_tablet_manager.cc:1403] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:30.718192 6284 raft_consensus.cc:359] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.718325 6284 raft_consensus.cc:385] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.718572 6284 raft_consensus.cc:740] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.718843 6284 consensus_queue.cc:260] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.719014 6284 raft_consensus.cc:399] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.719106 6284 raft_consensus.cc:493] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.719209 6284 raft_consensus.cc:3060] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.721728 6284 raft_consensus.cc:515] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.721938 6284 leader_election.cc:304] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8128523e364ce89100e9b6cfdb065f; no voters:
I20251212 08:53:30.722163 6284 leader_election.cc:290] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.722245 6288 raft_consensus.cc:2804] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.722450 6288 raft_consensus.cc:697] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Becoming Leader. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Running, Role: LEADER
I20251212 08:53:30.722776 6284 ts_tablet_manager.cc:1434] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:53:30.722733 6288 consensus_queue.cc:237] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.723066 6284 tablet_bootstrap.cc:492] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f: Bootstrap starting.
I20251212 08:53:30.724345 6284 tablet_bootstrap.cc:654] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.726303 6284 tablet_bootstrap.cc:492] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f: No bootstrap required, opened a new log
I20251212 08:53:30.726433 6284 ts_tablet_manager.cc:1403] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:30.726989 6284 raft_consensus.cc:359] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.727129 6284 raft_consensus.cc:385] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.727207 6284 raft_consensus.cc:740] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.727411 6284 consensus_queue.cc:260] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.727546 6284 raft_consensus.cc:399] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.727627 6284 raft_consensus.cc:493] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.727726 6284 raft_consensus.cc:3060] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.729805 6284 raft_consensus.cc:515] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.730052 6284 leader_election.cc:304] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8128523e364ce89100e9b6cfdb065f; no voters:
I20251212 08:53:30.730261 6284 leader_election.cc:290] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.730322 6286 raft_consensus.cc:2804] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.730839 6284 ts_tablet_manager.cc:1434] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20251212 08:53:30.731096 6284 tablet_bootstrap.cc:492] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f: Bootstrap starting.
I20251212 08:53:30.732190 6284 tablet_bootstrap.cc:654] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:30.733645 6284 tablet_bootstrap.cc:492] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f: No bootstrap required, opened a new log
I20251212 08:53:30.733788 6284 ts_tablet_manager.cc:1403] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:30.734318 6284 raft_consensus.cc:359] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.734454 6284 raft_consensus.cc:385] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:30.734546 6284 raft_consensus.cc:740] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Initialized, Role: FOLLOWER
I20251212 08:53:30.734822 6284 consensus_queue.cc:260] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.734977 6284 raft_consensus.cc:399] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:30.735067 6284 raft_consensus.cc:493] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:30.735167 6284 raft_consensus.cc:3060] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:30.735930 6286 raft_consensus.cc:697] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Becoming Leader. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Running, Role: LEADER
I20251212 08:53:30.736320 6286 consensus_queue.cc:237] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.737680 6284 raft_consensus.cc:515] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.737891 6284 leader_election.cc:304] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8128523e364ce89100e9b6cfdb065f; no voters:
I20251212 08:53:30.738140 6288 raft_consensus.cc:2804] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:30.738081 6284 leader_election.cc:290] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:30.738075 6128 catalog_manager.cc:5654] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8128523e364ce89100e9b6cfdb065f (127.0.208.1). New cstate: current_term: 1 leader_uuid: "8c8128523e364ce89100e9b6cfdb065f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:30.738795 6284 ts_tablet_manager.cc:1434] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:53:30.739334 6288 raft_consensus.cc:697] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Becoming Leader. State: Replica: 8c8128523e364ce89100e9b6cfdb065f, State: Running, Role: LEADER
I20251212 08:53:30.739604 6288 consensus_queue.cc:237] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } }
I20251212 08:53:30.743844 6128 catalog_manager.cc:5654] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8128523e364ce89100e9b6cfdb065f (127.0.208.1). New cstate: current_term: 1 leader_uuid: "8c8128523e364ce89100e9b6cfdb065f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:30.744156 6128 catalog_manager.cc:5654] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c8128523e364ce89100e9b6cfdb065f (127.0.208.1). New cstate: current_term: 1 leader_uuid: "8c8128523e364ce89100e9b6cfdb065f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8128523e364ce89100e9b6cfdb065f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36515 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:30.962278 6118 maintenance_manager.cc:419] P 278fbf43d3ed4e08ab6210169db2a0da: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=1.000000
I20251212 08:53:30.970098 6117 maintenance_manager.cc:643] P 278fbf43d3ed4e08ab6210169db2a0da: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.007s user 0.005s sys 0.000s Metrics: {"bytes_written":396,"delete_count":0,"lbm_write_time_us":224,"lbm_writes_lt_1ms":4,"reinsert_count":0,"update_count":4}
I20251212 08:53:30.971136 6118 maintenance_manager.cc:419] P 278fbf43d3ed4e08ab6210169db2a0da: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.120879
I20251212 08:53:31.001909 6117 maintenance_manager.cc:643] P 278fbf43d3ed4e08ab6210169db2a0da: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.030s user 0.015s sys 0.000s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2939,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":1774,"dirs.run_cpu_time_us":223,"dirs.run_wall_time_us":794,"lbm_read_time_us":423,"lbm_reads_lt_1ms":21,"lbm_write_time_us":406,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":1403,"threads_started":1,"update_count":4}
I20251212 08:53:31.116191 6275 maintenance_manager.cc:419] P 8c8128523e364ce89100e9b6cfdb065f: Scheduling FlushMRSOp(0904da7de5a44466a482b2af5c21c1e3): perf score=1.000000
I20251212 08:53:31.195144 6208 maintenance_manager.cc:643] P 8c8128523e364ce89100e9b6cfdb065f: FlushMRSOp(0904da7de5a44466a482b2af5c21c1e3) complete. Timing: real 0.079s user 0.043s sys 0.013s Metrics: {"bytes_written":4818,"cfile_init":1,"dirs.queue_time_us":32159,"dirs.run_cpu_time_us":210,"dirs.run_wall_time_us":856,"drs_written":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4,"lbm_write_time_us":835,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":250,"thread_start_us":32026,"threads_started":1}
I20251212 08:53:31.195966 6275 maintenance_manager.cc:419] P 8c8128523e364ce89100e9b6cfdb065f: Scheduling UndoDeltaBlockGCOp(0904da7de5a44466a482b2af5c21c1e3): 1705 bytes on disk
I20251212 08:53:31.196565 6208 maintenance_manager.cc:643] P 8c8128523e364ce89100e9b6cfdb065f: UndoDeltaBlockGCOp(0904da7de5a44466a482b2af5c21c1e3) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":123,"lbm_reads_lt_1ms":4}
I20251212 08:53:31.197108 6275 maintenance_manager.cc:419] P 8c8128523e364ce89100e9b6cfdb065f: Scheduling FlushMRSOp(1c2156c9f5a340ada54c8d574e85df3e): perf score=1.000000
I20251212 08:53:31.233966 6208 maintenance_manager.cc:643] P 8c8128523e364ce89100e9b6cfdb065f: FlushMRSOp(1c2156c9f5a340ada54c8d574e85df3e) complete. Timing: real 0.037s user 0.018s sys 0.003s Metrics: {"bytes_written":4816,"cfile_init":1,"dirs.queue_time_us":59,"dirs.run_cpu_time_us":204,"dirs.run_wall_time_us":1308,"drs_written":1,"lbm_read_time_us":122,"lbm_reads_lt_1ms":4,"lbm_write_time_us":876,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":250}
I20251212 08:53:31.234889 6275 maintenance_manager.cc:419] P 8c8128523e364ce89100e9b6cfdb065f: Scheduling UndoDeltaBlockGCOp(1c2156c9f5a340ada54c8d574e85df3e): 1705 bytes on disk
I20251212 08:53:31.235551 6208 maintenance_manager.cc:643] P 8c8128523e364ce89100e9b6cfdb065f: UndoDeltaBlockGCOp(1c2156c9f5a340ada54c8d574e85df3e) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":108,"lbm_reads_lt_1ms":4}
I20251212 08:53:31.236162 6275 maintenance_manager.cc:419] P 8c8128523e364ce89100e9b6cfdb065f: Scheduling FlushMRSOp(b75dae41f671452399b44e584630c12e): perf score=1.000000
I20251212 08:53:31.283051 6208 maintenance_manager.cc:643] P 8c8128523e364ce89100e9b6cfdb065f: FlushMRSOp(b75dae41f671452399b44e584630c12e) complete. Timing: real 0.047s user 0.032s sys 0.001s Metrics: {"bytes_written":4799,"cfile_init":1,"dirs.queue_time_us":57,"dirs.run_cpu_time_us":192,"dirs.run_wall_time_us":2763,"drs_written":1,"lbm_read_time_us":111,"lbm_reads_lt_1ms":4,"lbm_write_time_us":902,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":250}
I20251212 08:53:31.284315 6275 maintenance_manager.cc:419] P 8c8128523e364ce89100e9b6cfdb065f: Scheduling FlushMRSOp(f35c201cd6e743c5968e97d104a47719): perf score=1.000000
I20251212 08:53:31.375720 6208 maintenance_manager.cc:643] P 8c8128523e364ce89100e9b6cfdb065f: FlushMRSOp(f35c201cd6e743c5968e97d104a47719) complete. Timing: real 0.091s user 0.036s sys 0.000s Metrics: {"bytes_written":4826,"cfile_init":1,"dirs.queue_time_us":1752,"dirs.run_cpu_time_us":170,"dirs.run_wall_time_us":1272,"drs_written":1,"lbm_read_time_us":96,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1190,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":250}
I20251212 08:53:31.376585 6275 maintenance_manager.cc:419] P 8c8128523e364ce89100e9b6cfdb065f: Scheduling UndoDeltaBlockGCOp(f35c201cd6e743c5968e97d104a47719): 1705 bytes on disk
I20251212 08:53:31.377116 6208 maintenance_manager.cc:643] P 8c8128523e364ce89100e9b6cfdb065f: UndoDeltaBlockGCOp(f35c201cd6e743c5968e97d104a47719) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4}
I20251212 08:53:31.603196 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:31.622356 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:31.622677 832 tablet_replica.cc:333] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f: stopping tablet replica
I20251212 08:53:31.622870 832 raft_consensus.cc:2243] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:31.623077 832 raft_consensus.cc:2272] T f35c201cd6e743c5968e97d104a47719 P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:31.623980 832 tablet_replica.cc:333] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f: stopping tablet replica
I20251212 08:53:31.624135 832 raft_consensus.cc:2243] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:31.624344 832 raft_consensus.cc:2272] T 1c2156c9f5a340ada54c8d574e85df3e P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:31.625032 832 tablet_replica.cc:333] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f: stopping tablet replica
I20251212 08:53:31.625166 832 raft_consensus.cc:2243] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:31.625306 832 raft_consensus.cc:2272] T 0904da7de5a44466a482b2af5c21c1e3 P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:31.625918 832 tablet_replica.cc:333] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f: stopping tablet replica
I20251212 08:53:31.626044 832 raft_consensus.cc:2243] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:31.626178 832 raft_consensus.cc:2272] T b75dae41f671452399b44e584630c12e P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:31.626863 832 tablet_replica.cc:333] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f: stopping tablet replica
I20251212 08:53:31.626988 832 raft_consensus.cc:2243] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:31.627115 832 raft_consensus.cc:2272] T 15129d768dc0498aaeb3ae7bc0207f9e P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:31.627652 832 tablet_replica.cc:333] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f: stopping tablet replica
I20251212 08:53:31.627779 832 raft_consensus.cc:2243] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:31.627893 832 raft_consensus.cc:2272] T 03b5fd7917574323bd0781409c5cf7c0 P 8c8128523e364ce89100e9b6cfdb065f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:31.643107 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:31.650592 832 master.cc:561] Master@127.0.208.62:32773 shutting down...
I20251212 08:53:31.663012 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:31.663292 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:31.663410 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 278fbf43d3ed4e08ab6210169db2a0da: stopping tablet replica
I20251212 08:53:31.677390 832 master.cc:583] Master@127.0.208.62:32773 shutdown complete.
[ OK ] TableKeyRangeTest.TestGetTableKeyRange (1276 ms)
[----------] 1 test from TableKeyRangeTest (1276 ms total)
[----------] 1 test from MultiTServerClientTest
[ RUN ] MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers
I20251212 08:53:31.694249 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:45467
I20251212 08:53:31.694892 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:31.697700 6457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:31.697739 6458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:31.698815 832 server_base.cc:1047] running on GCE node
W20251212 08:53:31.698822 6460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:31.699252 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:31.699316 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:31.699355 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529611699352 us; error 0 us; skew 500 ppm
I20251212 08:53:31.699514 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:31.700822 832 webserver.cc:492] Webserver started at http://127.0.208.62:38365/ using document root <none> and password file <none>
I20251212 08:53:31.701061 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:31.701128 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:31.701247 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:31.702041 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "f5ed567a363645ad9cbdcea448ad3750"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:31.705229 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20251212 08:53:31.707070 6465 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:31.707472 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:31.707564 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "f5ed567a363645ad9cbdcea448ad3750"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:31.707676 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:31.718537 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:31.719022 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:31.733234 6516 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:45467 every 8 connection(s)
I20251212 08:53:31.733234 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:45467
I20251212 08:53:31.734742 6517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:31.741032 6517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750: Bootstrap starting.
I20251212 08:53:31.742496 6517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:31.744135 6517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750: No bootstrap required, opened a new log
I20251212 08:53:31.744856 6517 raft_consensus.cc:359] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ed567a363645ad9cbdcea448ad3750" member_type: VOTER }
I20251212 08:53:31.745010 6517 raft_consensus.cc:385] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:31.745076 6517 raft_consensus.cc:740] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5ed567a363645ad9cbdcea448ad3750, State: Initialized, Role: FOLLOWER
I20251212 08:53:31.745277 6517 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ed567a363645ad9cbdcea448ad3750" member_type: VOTER }
I20251212 08:53:31.745420 6517 raft_consensus.cc:399] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:53:31.745486 6517 raft_consensus.cc:493] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:53:31.745560 6517 raft_consensus.cc:3060] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:31.748548 6517 raft_consensus.cc:515] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ed567a363645ad9cbdcea448ad3750" member_type: VOTER }
I20251212 08:53:31.748771 6517 leader_election.cc:304] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5ed567a363645ad9cbdcea448ad3750; no voters:
I20251212 08:53:31.749192 6517 leader_election.cc:290] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:53:31.749301 6520 raft_consensus.cc:2804] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:31.750026 6520 raft_consensus.cc:697] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 1 LEADER]: Becoming Leader. State: Replica: f5ed567a363645ad9cbdcea448ad3750, State: Running, Role: LEADER
I20251212 08:53:31.750054 6517 sys_catalog.cc:565] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:53:31.750272 6520 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ed567a363645ad9cbdcea448ad3750" member_type: VOTER }
I20251212 08:53:31.751080 6521 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5ed567a363645ad9cbdcea448ad3750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ed567a363645ad9cbdcea448ad3750" member_type: VOTER } }
I20251212 08:53:31.751216 6521 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:31.751574 6522 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5ed567a363645ad9cbdcea448ad3750. Latest consensus state: current_term: 1 leader_uuid: "f5ed567a363645ad9cbdcea448ad3750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5ed567a363645ad9cbdcea448ad3750" member_type: VOTER } }
I20251212 08:53:31.751708 6522 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [sys.catalog]: This master's current role is: LEADER
I20251212 08:53:31.775899 6525 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:53:31.777954 6525 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:53:31.782269 6525 catalog_manager.cc:1357] Generated new cluster ID: faef2024522b4ecfadc282b765dd1e4c
I20251212 08:53:31.782357 6525 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:53:31.789726 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:53:31.793008 6525 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:53:31.794087 6525 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:53:31.808833 6525 catalog_manager.cc:6027] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750: Generated new TSK 0
I20251212 08:53:31.809075 6525 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:53:31.822513 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:31.825440 6539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:31.825551 6540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:31.826359 6542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:31.826596 832 server_base.cc:1047] running on GCE node
I20251212 08:53:31.826848 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:31.826901 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:31.826936 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529611826932 us; error 0 us; skew 500 ppm
I20251212 08:53:31.827095 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:31.828459 832 webserver.cc:492] Webserver started at http://127.0.208.1:36883/ using document root <none> and password file <none>
I20251212 08:53:31.828680 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:31.828749 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:31.828882 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:31.829571 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:31.832073 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:53:31.833719 6547 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:31.834088 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:53:31.834177 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:31.834301 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:31.849856 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:31.850376 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:31.850981 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:31.851967 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:31.852022 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:31.852085 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:31.852130 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:31.873777 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35903
I20251212 08:53:31.873778 6609 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35903 every 8 connection(s)
I20251212 08:53:31.875576 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:31.890236 6614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:31.890492 6615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:31.898144 6610 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:31.898365 6610 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:31.898921 6610 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:31.900136 6482 ts_manager.cc:194] Registered new tserver with Master: 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:31.902204 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53584
I20251212 08:53:31.905546 832 server_base.cc:1047] running on GCE node
W20251212 08:53:31.905553 6617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:31.906045 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:31.906116 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:31.906139 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529611906137 us; error 0 us; skew 500 ppm
I20251212 08:53:31.906296 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:31.907527 832 webserver.cc:492] Webserver started at http://127.0.208.2:36849/ using document root <none> and password file <none>
I20251212 08:53:31.907733 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:31.907796 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:31.907917 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:31.908584 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/instance:
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:31.911051 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:31.913261 6622 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:31.913697 832 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:31.913862 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:31.914075 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:31.932574 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:31.933043 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:31.933581 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:31.934422 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:31.934484 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:31.934543 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:31.934589 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:31.948670 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:42203
I20251212 08:53:31.948711 6684 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:42203 every 8 connection(s)
I20251212 08:53:31.950146 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:31.952764 6688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:31.953053 6689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:31.953555 6691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:31.955484 832 server_base.cc:1047] running on GCE node
I20251212 08:53:31.955785 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:31.955956 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:31.956037 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529611956033 us; error 0 us; skew 500 ppm
I20251212 08:53:31.956209 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:31.956303 6685 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:31.956470 6685 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:31.956868 6685 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:31.957434 832 webserver.cc:492] Webserver started at http://127.0.208.3:35191/ using document root <none> and password file <none>
I20251212 08:53:31.957631 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:31.957700 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:31.957752 6482 ts_manager.cc:194] Registered new tserver with Master: 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:31.957814 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:31.958391 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/instance:
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:31.958484 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53600
I20251212 08:53:31.960844 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:31.962316 6696 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:31.962708 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:31.962800 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:31.962890 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:32.007967 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:32.008505 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:32.009070 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:32.009934 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:32.009992 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:32.010048 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:32.010097 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:32.023671 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39569
I20251212 08:53:32.023710 6758 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39569 every 8 connection(s)
I20251212 08:53:32.025213 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:32.028016 6763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:32.027976 6762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:32.028880 6765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:32.029533 832 server_base.cc:1047] running on GCE node
I20251212 08:53:32.029789 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:32.029850 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:32.029891 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529612029887 us; error 0 us; skew 500 ppm
I20251212 08:53:32.030360 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:32.031697 832 webserver.cc:492] Webserver started at http://127.0.208.4:44337/ using document root <none> and password file <none>
I20251212 08:53:32.031708 6759 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:32.031857 6759 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:32.031903 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:32.031980 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:53:32.032090 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:53:32.032214 6759 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:32.032757 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/instance:
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:53:32.033057 6482 ts_manager.cc:194] Registered new tserver with Master: f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:32.033789 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53614
I20251212 08:53:32.035324 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:53:32.037009 6770 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:32.037330 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:32.037436 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:53:32.037534 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:32.047845 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:32.048316 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:32.048782 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:32.049590 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:53:32.049651 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:32.049732 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:53:32.049772 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:32.061041 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.4:45411
I20251212 08:53:32.061074 6832 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.4:45411 every 8 connection(s)
I20251212 08:53:32.067229 6833 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:32.067376 6833 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:32.067720 6833 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:32.068539 6482 ts_manager.cc:194] Registered new tserver with Master: 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411)
I20251212 08:53:32.068574 832 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.006764672s
I20251212 08:53:32.069250 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53620
I20251212 08:53:32.079749 6482 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53624:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
I20251212 08:53:32.111925 6575 tablet_service.cc:1505] Processing CreateTablet for tablet fd6fbc8b4b1d424cb5a077390cf3314d (DEFAULT_TABLE table=client-testtb [id=848c343b3b6a4b0c87337fd2b88d113c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251212 08:53:32.112498 6575 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd6fbc8b4b1d424cb5a077390cf3314d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:32.112377 6798 tablet_service.cc:1505] Processing CreateTablet for tablet 1231948ca74843e28105be29550fbdf8 (DEFAULT_TABLE table=client-testtb [id=848c343b3b6a4b0c87337fd2b88d113c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251212 08:53:32.113101 6798 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1231948ca74843e28105be29550fbdf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:32.115311 6650 tablet_service.cc:1505] Processing CreateTablet for tablet fd6fbc8b4b1d424cb5a077390cf3314d (DEFAULT_TABLE table=client-testtb [id=848c343b3b6a4b0c87337fd2b88d113c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251212 08:53:32.115653 6649 tablet_service.cc:1505] Processing CreateTablet for tablet 1231948ca74843e28105be29550fbdf8 (DEFAULT_TABLE table=client-testtb [id=848c343b3b6a4b0c87337fd2b88d113c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251212 08:53:32.115679 6650 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd6fbc8b4b1d424cb5a077390cf3314d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:32.116412 6649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1231948ca74843e28105be29550fbdf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:32.116412 6574 tablet_service.cc:1505] Processing CreateTablet for tablet 1231948ca74843e28105be29550fbdf8 (DEFAULT_TABLE table=client-testtb [id=848c343b3b6a4b0c87337fd2b88d113c]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251212 08:53:32.117489 6574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1231948ca74843e28105be29550fbdf8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:32.127578 6849 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:32.129186 6849 tablet_bootstrap.cc:654] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:32.131390 6849 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: No bootstrap required, opened a new log
I20251212 08:53:32.131524 6849 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:32.132011 6850 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:32.132175 6849 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.132380 6849 raft_consensus.cc:385] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:32.132447 6849 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.132702 6849 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.134738 6850 tablet_bootstrap.cc:654] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:32.139635 6849 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.008s user 0.005s sys 0.004s
I20251212 08:53:32.139844 6610 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:32.142761 6849 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:32.143781 6850 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: No bootstrap required, opened a new log
I20251212 08:53:32.143919 6850 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20251212 08:53:32.144502 6849 tablet_bootstrap.cc:654] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:32.144578 6850 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.144785 6850 raft_consensus.cc:385] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:32.144845 6850 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.145071 6850 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.145673 6685 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:32.147363 6850 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:32.147629 6850 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:32.149001 6850 tablet_bootstrap.cc:654] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:32.149081 6854 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap starting.
I20251212 08:53:32.150837 6854 tablet_bootstrap.cc:654] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:32.152861 6854 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: No bootstrap required, opened a new log
I20251212 08:53:32.152987 6854 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20251212 08:53:32.153020 6724 tablet_service.cc:1505] Processing CreateTablet for tablet fd6fbc8b4b1d424cb5a077390cf3314d (DEFAULT_TABLE table=client-testtb [id=848c343b3b6a4b0c87337fd2b88d113c]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251212 08:53:32.153195 6849 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: No bootstrap required, opened a new log
I20251212 08:53:32.153307 6849 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20251212 08:53:32.153411 6724 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd6fbc8b4b1d424cb5a077390cf3314d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:53:32.153671 6854 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.153831 6849 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.153985 6849 raft_consensus.cc:385] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:32.154049 6849 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.153839 6854 raft_consensus.cc:385] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:32.154150 6854 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.154280 6849 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.154336 6854 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.154752 6849 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20251212 08:53:32.155063 6850 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: No bootstrap required, opened a new log
I20251212 08:53:32.155198 6850 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20251212 08:53:32.156008 6850 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.156196 6850 raft_consensus.cc:385] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:32.156260 6850 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.156589 6850 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.157204 6850 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:32.160535 6833 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:32.160562 6854 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent starting tablet: real 0.007s user 0.000s sys 0.007s
I20251212 08:53:32.162036 6860 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap starting.
I20251212 08:53:32.163472 6860 tablet_bootstrap.cc:654] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Neither blocks nor log segments found. Creating new log.
I20251212 08:53:32.169430 6860 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: No bootstrap required, opened a new log
I20251212 08:53:32.169567 6860 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20251212 08:53:32.170156 6860 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.170315 6860 raft_consensus.cc:385] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:53:32.170373 6860 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.170576 6860 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.171177 6860 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:53:32.171238 6759 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:32.214015 6852 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:32.214211 6852 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.215030 6852 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:32.221244 6734 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:32.221244 6660 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:32.221652 6734 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 0.
I20251212 08:53:32.221652 6660 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 0.
I20251212 08:53:32.222172 6551 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:32.222426 6852 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:32.222506 6852 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:32.222580 6852 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:32.224642 6852 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.225287 6852 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 1 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:32.225790 6660 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:32.225790 6734 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:32.226006 6734 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:32.226006 6660 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:32.229322 6734 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 1.
I20251212 08:53:32.229322 6660 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 1.
I20251212 08:53:32.229802 6550 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:32.230084 6852 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:32.230479 6852 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 1 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:32.230729 6852 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.233525 6482 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:32.246460 6855 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:32.246615 6855 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.247313 6855 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:32.253665 6808 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:32.253665 6585 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:32.253965 6585 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 0.
I20251212 08:53:32.253965 6808 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 0.
I20251212 08:53:32.254431 6626 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:32.254683 6855 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:53:32.254761 6855 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:32.254843 6855 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:32.256879 6855 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.257351 6855 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 1 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:32.257793 6808 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:32.257793 6585 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:32.258072 6808 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:32.258072 6585 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:53:32.261138 6585 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 1.
I20251212 08:53:32.261138 6808 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 1.
I20251212 08:53:32.261641 6626 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:32.261879 6855 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:53:32.262248 6855 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:53:32.262723 6855 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.265282 6482 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2). New cstate: current_term: 1 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:32.297891 6734 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 1 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:32.297891 6660 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:32.298539 6864 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:32.298731 6852 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:32.307013 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:53:32.309454 6872 mvcc.cc:204] Tried to move back new op lower bound from 7231609291967229952 to 7231609291699388416. Current Snapshot: MvccSnapshot[applied={T|T < 7231609291967229952}]
I20251212 08:53:32.316164 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:32.316690 832 tablet_replica.cc:333] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: stopping tablet replica
I20251212 08:53:32.316852 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:32.316982 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:32.317539 832 tablet_replica.cc:333] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: stopping tablet replica
I20251212 08:53:32.317644 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:32.317884 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:32.322781 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:53:32.326498 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:32.328908 6878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:32.329165 6879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:32.329588 6881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:32.329828 832 server_base.cc:1047] running on GCE node
I20251212 08:53:32.330046 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:32.330101 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:32.330165 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529612330155 us; error 0 us; skew 500 ppm
I20251212 08:53:32.330343 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:32.331625 832 webserver.cc:492] Webserver started at http://127.0.208.1:36883/ using document root <none> and password file <none>
I20251212 08:53:32.331849 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:32.331907 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:32.333973 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:32.335407 6886 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:32.335762 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:32.335863 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:32.335981 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:32.355731 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:32.356221 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:32.356802 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:32.357967 6893 ts_tablet_manager.cc:548] Loading tablet metadata (0/2 complete)
I20251212 08:53:32.362361 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:32.362463 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20251212 08:53:32.362553 832 ts_tablet_manager.cc:600] Registering tablets (0/2 complete)
I20251212 08:53:32.364770 6893 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:32.366281 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:32.366346 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:32.368248 6893 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:32.368666 6893 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:32.368981 6893 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:32.369761 6893 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.369930 6893 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.370215 6893 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.372776 6893 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.004s user 0.002s sys 0.002s
I20251212 08:53:32.373009 6893 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:32.378508 6893 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:32.378899 6893 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:32.379122 6893 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20251212 08:53:32.379729 6893 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.379877 6893 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.380153 6893 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.380604 6893 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.001s user 0.003s sys 0.000s
I20251212 08:53:32.382020 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35903
I20251212 08:53:32.382073 6952 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35903 every 8 connection(s)
I20251212 08:53:32.388679 6953 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:32.388811 6953 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:32.389168 6953 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:32.390543 6480 ts_manager.cc:194] Re-registered known tserver with Master: 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:32.391254 6808 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 1 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:32.391825 6867 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:32.392633 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53638
I20251212 08:53:32.398494 832 tablet_server.cc:178] TabletServer@127.0.208.2:0 shutting down...
I20251212 08:53:32.407078 6927 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:53:32.407828 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:32.407842 6855 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:53:32.408167 832 tablet_replica.cc:333] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: stopping tablet replica
I20251212 08:53:32.408408 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:32.408638 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:32.409611 832 tablet_replica.cc:333] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: stopping tablet replica
I20251212 08:53:32.409746 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:53:32.409983 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:32.413002 6953 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:32.417191 6626 proxy.cc:239] Call had error, refreshing address and retrying: Service unavailable: reactor is shutting down (error 108) [suppressed 28 similar messages]
W20251212 08:53:32.422817 6888 connection.cc:441] server connection from 127.0.0.1:53342 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53342 (request call id 1) could send its response
I20251212 08:53:32.430013 832 tablet_server.cc:195] TabletServer@127.0.208.2:0 shutdown complete.
I20251212 08:53:32.433645 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:32.435957 6964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:32.435964 6965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:32.437052 6967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:32.437301 832 server_base.cc:1047] running on GCE node
I20251212 08:53:32.437639 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:32.437698 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:32.437732 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529612437730 us; error 0 us; skew 500 ppm
I20251212 08:53:32.437834 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:32.439085 832 webserver.cc:492] Webserver started at http://127.0.208.2:36849/ using document root <none> and password file <none>
I20251212 08:53:32.439256 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:32.439299 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:32.441047 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:32.442256 6972 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:32.442631 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:32.442718 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:32.442830 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:32.460269 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:32.460781 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:32.461282 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:32.467330 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:32.467417 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20251212 08:53:32.469497 6979 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:32.471215 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:32.471269 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20251212 08:53:32.474588 6979 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:32.474947 6979 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:32.475174 6979 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20251212 08:53:32.475718 6979 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.475864 6979 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.476145 6979 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:32.476955 6979 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.004s
I20251212 08:53:32.477174 6979 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:32.482089 6979 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:32.482491 6979 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:32.482715 6979 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20251212 08:53:32.483397 6979 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.483573 6979 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:32.483786 6979 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:32.484190 6979 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
I20251212 08:53:32.487849 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:42203
I20251212 08:53:32.487870 7038 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:42203 every 8 connection(s)
I20251212 08:53:32.495376 7039 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:32.495515 7039 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:32.495854 7039 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
W20251212 08:53:32.496012 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:32.496960 6480 ts_manager.cc:194] Re-registered known tserver with Master: 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:32.498198 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53646
W20251212 08:53:32.505390 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:32.512741 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.519582 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.529361 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:32.537220 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.549295 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.560065 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:32.574419 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.588804 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.604857 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:32.621747 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.637802 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.654691 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:32.670603 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.690258 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.708226 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:32.730232 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.751760 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.775858 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:32.801896 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.830164 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.855139 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:32.880216 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.911453 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.939503 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:32.967478 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:32.998154 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.032192 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:33.067234 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.103639 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.139564 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:33.176726 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.213236 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.251674 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:33.291719 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.333619 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.374780 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:33.414876 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.460253 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:33.499418 7039 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:33.506214 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:33.550390 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.599396 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.645627 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:33.694937 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.746378 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:33.798861 6788 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:35738: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:33.810597 7043 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:33.810792 7043 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:33.811508 7043 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:33.830106 7014 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:33.830340 6927 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:33.830389 7014 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 1.
I20251212 08:53:33.830492 6927 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 1.
I20251212 08:53:33.830868 6699 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:33.831162 7043 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251212 08:53:33.831248 7043 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:33.831342 7043 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 1 FOLLOWER]: Advancing to term 2
I20251212 08:53:33.834125 7043 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:33.834718 7043 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 2 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:33.835278 6927 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:33.835283 7014 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:33.835510 7014 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Advancing to term 2
I20251212 08:53:33.835513 6927 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Advancing to term 2
I20251212 08:53:33.838804 6927 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 2.
I20251212 08:53:33.838804 7014 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 2.
I20251212 08:53:33.839260 6700 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:33.839576 7043 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Leader election won for term 2
I20251212 08:53:33.839970 7043 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 LEADER]: Becoming Leader. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Running, Role: LEADER
I20251212 08:53:33.840216 7043 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:33.842765 6480 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 reported cstate change: term changed from 1 to 2, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3). New cstate: current_term: 2 leader_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: HEALTHY } } }
W20251212 08:53:33.851785 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:33.898126 7054 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:33.898327 7054 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:33.899058 7054 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:33.904930 7014 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:33.904930 6927 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:33.905220 7014 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 1.
W20251212 08:53:33.905392 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:33.905537 6773 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters:
I20251212 08:53:33.905755 7054 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251212 08:53:33.905829 7054 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 1 FOLLOWER]: Starting leader election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:33.905900 7054 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 1 FOLLOWER]: Advancing to term 2
I20251212 08:53:33.908034 7054 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:33.908668 7054 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 2 election: Requested vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:33.909168 6927 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:33.909152 7014 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:33.909371 7014 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 1 FOLLOWER]: Advancing to term 2
I20251212 08:53:33.911654 7014 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 2.
I20251212 08:53:33.912101 6773 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:33.912427 7054 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 2 FOLLOWER]: Leader election won for term 2
I20251212 08:53:33.912859 7054 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 2 LEADER]: Becoming Leader. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Running, Role: LEADER
I20251212 08:53:33.913158 7054 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:33.913686 7058 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:33.913818 7058 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:33.915670 7058 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:33.917197 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed reported cstate change: term changed from 1 to 2, leader changed from 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2) to 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4). New cstate: current_term: 2 leader_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:33.922729 6808 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:33.922729 7014 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:33.923027 7014 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 in current term 2: Already voted for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in this term.
I20251212 08:53:33.923358 6889 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed
I20251212 08:53:33.923580 7058 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 1 FOLLOWER]: Advancing to term 2
I20251212 08:53:33.926050 7058 raft_consensus.cc:2749] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251212 08:53:33.960537 6927 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Refusing update from remote peer 808a87a7594d4fa0b5f8d1c78aebb8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251212 08:53:33.960541 7014 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Refusing update from remote peer 808a87a7594d4fa0b5f8d1c78aebb8ed: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20251212 08:53:33.961154 7057 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251212 08:53:33.961344 7054 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20251212 08:53:33.970860 832 tablet_server.cc:178] TabletServer@127.0.208.3:0 shutting down...
I20251212 08:53:33.987426 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:33.987910 832 tablet_replica.cc:333] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: stopping tablet replica
I20251212 08:53:33.988256 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 LEADER]: Raft consensus shutting down.
I20251212 08:53:33.988806 832 pending_rounds.cc:70] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Trying to abort 1 pending ops.
I20251212 08:53:33.989017 832 pending_rounds.cc:77] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Aborting op as it isn't in flight: id { term: 2 index: 3 } timestamp: 7231609298292039680 op_type: NO_OP noop_request { }
I20251212 08:53:33.989133 832 raft_consensus.cc:2889] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251212 08:53:33.989207 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:34.004027 832 tablet_server.cc:195] TabletServer@127.0.208.3:0 shutdown complete.
I20251212 08:53:34.008152 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:34.010907 7073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:34.010993 7074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:34.012081 7076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:34.012570 832 server_base.cc:1047] running on GCE node
I20251212 08:53:34.012817 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:34.012872 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:34.012897 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529614012894 us; error 0 us; skew 500 ppm
I20251212 08:53:34.013033 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:34.014457 832 webserver.cc:492] Webserver started at http://127.0.208.3:35191/ using document root <none> and password file <none>
I20251212 08:53:34.014678 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:34.014731 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:34.016716 832 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:34.018071 7081 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:34.018450 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:34.018541 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:34.018656 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:34.026813 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:34.027356 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:34.027897 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:34.028952 7088 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:34.031426 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:34.031525 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:34.031641 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:34.033619 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:34.033679 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:34.033768 7088 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap starting.
I20251212 08:53:34.040755 7088 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251212 08:53:34.041488 7088 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap complete.
I20251212 08:53:34.041800 7088 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20251212 08:53:34.042869 7088 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:34.043241 7088 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Initialized, Role: FOLLOWER
I20251212 08:53:34.043558 7088 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:34.044405 7088 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent starting tablet: real 0.002s user 0.002s sys 0.002s
I20251212 08:53:34.051173 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39569
I20251212 08:53:34.051198 7146 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39569 every 8 connection(s)
W20251212 08:53:34.053638 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:34.058153 7147 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:34.058301 7147 heartbeater.cc:461] Registering TS with master...
W20251212 08:53:34.058310 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:34.058686 7147 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:34.059718 6482 ts_manager.cc:194] Re-registered known tserver with Master: f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:34.060840 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38232
W20251212 08:53:34.071396 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.079627 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.086230 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.093159 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.104931 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.114872 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.128047 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.143532 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.158607 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.174964 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.190799 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.207062 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.223481 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.245067 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.265309 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.285501 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.309897 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.330878 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.353015 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.380609 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.405019 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.434158 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.462678 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.490732 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.521646 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.551752 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.585681 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.619776 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.657589 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.691799 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.725979 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.762594 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.801967 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.842077 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.881616 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.921823 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:34.962056 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.008790 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.054658 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:35.062002 7147 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:35.097837 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.145864 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.191982 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.240919 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.291579 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.340837 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.391058 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.442488 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.493856 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.546824 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.601586 7101 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43876: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.655637 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:35.699108 7155 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:35.699324 7155 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:35.700079 7155 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:35.713639 7014 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
W20251212 08:53:35.713730 6906 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:53352: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:35.713821 6927 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:35.713903 7014 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 2.
I20251212 08:53:35.714089 6927 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 2.
I20251212 08:53:35.714440 7084 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:35.714715 7155 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251212 08:53:35.714790 7155 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:35.714835 7155 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 2 FOLLOWER]: Advancing to term 3
I20251212 08:53:35.716974 7155 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:35.717548 7155 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 3 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:35.717954 7014 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:35.717971 6927 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:35.718114 7014 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Advancing to term 3
I20251212 08:53:35.718132 6927 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Advancing to term 3
I20251212 08:53:35.720250 6927 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 3.
I20251212 08:53:35.720261 7014 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 3.
I20251212 08:53:35.720702 7085 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:35.720939 7155 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 3 FOLLOWER]: Leader election won for term 3
I20251212 08:53:35.721387 7155 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 3 LEADER]: Becoming Leader. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Running, Role: LEADER
I20251212 08:53:35.721652 7155 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:35.724083 6482 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 reported cstate change: term changed from 2 to 3. New cstate: current_term: 3 leader_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:35.774468 6927 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20251212 08:53:35.774468 7014 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20251212 08:53:35.775070 7160 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20251212 08:53:35.775321 7155 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20251212 08:53:35.784780 832 tablet_server.cc:178] TabletServer@127.0.208.4:0 shutting down...
I20251212 08:53:35.809433 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:35.809827 832 tablet_replica.cc:333] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: stopping tablet replica
I20251212 08:53:35.810016 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 2 LEADER]: Raft consensus shutting down.
I20251212 08:53:35.810372 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 2 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:35.824545 832 tablet_server.cc:195] TabletServer@127.0.208.4:0 shutdown complete.
I20251212 08:53:35.827812 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:35.830379 7177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:35.830624 7178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:35.831053 7180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:35.831348 832 server_base.cc:1047] running on GCE node
I20251212 08:53:35.832186 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:35.832265 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:35.832355 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529615832333 us; error 0 us; skew 500 ppm
I20251212 08:53:35.832513 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:35.833709 832 webserver.cc:492] Webserver started at http://127.0.208.4:44337/ using document root <none> and password file <none>
I20251212 08:53:35.833915 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:35.833989 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:35.835770 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:35.837039 7185 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:35.837378 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:35.837491 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:53:35.837620 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:35.844918 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:35.845418 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:35.845918 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:35.847045 7192 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:35.849692 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:35.849795 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:35.849912 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:35.851972 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:35.852049 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20251212 08:53:35.852255 7192 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap starting.
I20251212 08:53:35.861019 7192 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=2 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:35.861694 7192 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap complete.
I20251212 08:53:35.862058 7192 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.003s
I20251212 08:53:35.862943 7192 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:35.863204 7192 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Initialized, Role: FOLLOWER
I20251212 08:53:35.863535 7192 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:35.864431 7192 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20251212 08:53:35.865878 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.4:45411
I20251212 08:53:35.865906 7250 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.4:45411 every 8 connection(s)
I20251212 08:53:35.873545 832 tablet_server.cc:178] TabletServer@127.0.208.1:35903 shutting down...
I20251212 08:53:35.874557 7251 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:35.875216 7251 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:35.875679 7251 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:35.876888 6480 ts_manager.cc:194] Re-registered known tserver with Master: 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411)
I20251212 08:53:35.878597 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38246
I20251212 08:53:35.886945 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:35.887233 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:35.887423 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:35.887637 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:35.888448 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:35.888573 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:35.888760 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:35.897701 832 tablet_server.cc:195] TabletServer@127.0.208.1:35903 shutdown complete.
I20251212 08:53:35.901594 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:35.904218 7256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:35.904357 7255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:35.904698 7258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:35.905030 832 server_base.cc:1047] running on GCE node
I20251212 08:53:35.905241 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:35.905385 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:35.905460 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529615905442 us; error 0 us; skew 500 ppm
I20251212 08:53:35.905628 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:35.906961 832 webserver.cc:492] Webserver started at http://127.0.208.1:36883/ using document root <none> and password file <none>
I20251212 08:53:35.907212 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:35.907286 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:35.909180 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:35.910531 7263 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:35.910918 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:35.911017 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:35.911132 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:35.923041 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:35.923542 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:35.924077 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:35.930068 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:35.930188 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20251212 08:53:35.932404 7270 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:35.934047 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:35.934145 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.000s sys 0.002s
I20251212 08:53:35.940146 7270 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=2 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:35.940783 7270 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:35.941066 7270 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20251212 08:53:35.941931 7270 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:35.942126 7270 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:35.942399 7270 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:35.943130 7270 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:35.943388 7270 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:35.950439 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35903
I20251212 08:53:35.950439 7329 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35903 every 8 connection(s)
I20251212 08:53:35.952144 7270 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:35.952713 7270 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:35.953048 7270 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20251212 08:53:35.953962 7270 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:35.954164 7270 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:35.954444 7270 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:35.954893 7270 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20251212 08:53:35.959190 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:35.959204 7330 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:35.959412 7330 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:35.959800 7330 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:35.961093 6480 ts_manager.cc:194] Re-registered known tserver with Master: 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:35.962852 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38260
I20251212 08:53:35.964047 7330 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:35.967911 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.976146 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:35.984404 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:35.992411 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.003444 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.012902 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.024791 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.036788 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.052174 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.064312 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.080760 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.098527 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.115532 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.131819 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.153682 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.175774 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.198899 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.222806 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.245607 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.271788 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.299055 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.323102 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.350200 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.382300 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.413731 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.444845 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.478909 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.510027 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.541064 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.577744 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.614751 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.650995 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.689677 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.728786 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.766538 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.808869 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.850998 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:36.880230 7251 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:36.895165 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:36.939648 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:36.983822 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.031039 6992 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41304: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:37.067315 7337 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:53:37.067582 7337 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:37.068413 7337 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
W20251212 08:53:37.078570 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:37.090315 7304 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:37.090327 7225 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:37.090593 7304 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 2.
I20251212 08:53:37.090618 7225 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 2.
I20251212 08:53:37.091035 6976 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:37.091329 7337 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251212 08:53:37.091436 7337 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:53:37.091521 7337 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 2 FOLLOWER]: Advancing to term 3
I20251212 08:53:37.093598 7337 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:37.094170 7337 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 3 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:37.094637 7304 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:37.094659 7225 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:37.094808 7304 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 2 FOLLOWER]: Advancing to term 3
I20251212 08:53:37.094815 7225 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 2 FOLLOWER]: Advancing to term 3
I20251212 08:53:37.096882 7304 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 3.
I20251212 08:53:37.096920 7225 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 3.
I20251212 08:53:37.097287 6974 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters:
I20251212 08:53:37.097572 7337 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Leader election won for term 3
I20251212 08:53:37.097957 7337 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:53:37.098217 7337 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:37.100878 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 2 to 3, leader changed from 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4) to 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2). New cstate: current_term: 3 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } }
W20251212 08:53:37.125028 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:37.175339 7225 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 3 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20251212 08:53:37.175339 7304 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20251212 08:53:37.176043 7342 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251212 08:53:37.176271 7337 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20251212 08:53:37.180907 7348 mvcc.cc:204] Tried to move back new op lower bound from 7231609311944691712 to 7231609311636361216. Current Snapshot: MvccSnapshot[applied={T|T < 7231609311944691712}]
I20251212 08:53:37.191308 832 tablet_server.cc:178] TabletServer@127.0.208.2:42203 shutting down...
I20251212 08:53:37.201102 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:37.201438 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:37.201619 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:37.201828 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:37.202433 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:37.202554 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 LEADER]: Raft consensus shutting down.
I20251212 08:53:37.202836 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:37.216769 832 tablet_server.cc:195] TabletServer@127.0.208.2:42203 shutdown complete.
I20251212 08:53:37.220430 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:37.223011 7358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:37.223085 7359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:37.223694 832 server_base.cc:1047] running on GCE node
W20251212 08:53:37.223681 7361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:37.224093 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:37.224149 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:37.224174 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529617224170 us; error 0 us; skew 500 ppm
I20251212 08:53:37.224277 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:37.225536 832 webserver.cc:492] Webserver started at http://127.0.208.2:36849/ using document root <none> and password file <none>
I20251212 08:53:37.225723 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:37.225768 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:37.227608 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:37.229039 7366 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:37.229436 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:37.229537 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:37.229669 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:37.262986 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:37.263525 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:37.264030 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:37.265127 7373 ts_tablet_manager.cc:548] Loading tablet metadata (0/2 complete)
I20251212 08:53:37.269901 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:37.270023 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20251212 08:53:37.270192 832 ts_tablet_manager.cc:600] Registering tablets (0/2 complete)
I20251212 08:53:37.272449 7373 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:37.274237 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:37.274329 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:37.281682 7373 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:37.282116 7373 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:37.282389 7373 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20251212 08:53:37.283156 7373 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:37.283349 7373 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:37.283605 7373 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:37.285327 7373 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20251212 08:53:37.285641 7373 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:37.293581 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:42203
I20251212 08:53:37.293601 7432 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:42203 every 8 connection(s)
I20251212 08:53:37.297844 7373 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:37.298842 7373 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:37.299448 7373 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.003s
I20251212 08:53:37.300741 7373 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:37.301106 7373 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:37.301486 7373 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:37.302368 7373 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:37.315294 7433 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:37.315481 7433 heartbeater.cc:461] Registering TS with master...
W20251212 08:53:37.315521 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:37.315845 7433 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:37.317075 6482 ts_manager.cc:194] Re-registered known tserver with Master: 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:37.318455 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38262
I20251212 08:53:37.319562 7433 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:37.322173 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.328895 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.336510 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.345597 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.355772 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.368538 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.379803 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.393965 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.408916 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.421839 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.437004 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.452611 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.471695 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.490836 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.513137 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.534123 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.554243 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.576820 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.601061 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.623095 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.649708 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.674957 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.704173 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.735473 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.763662 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.791766 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.823001 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.857141 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:37.892324 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.929637 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:37.962756 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:38.000209 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.040894 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.080173 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:38.118320 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.161648 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.203661 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:38.248045 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.292162 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.336576 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:38.381718 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.430266 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.477810 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:38.525019 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.575794 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.625958 7205 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60622: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:38.676389 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:38.682519 7440 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:38.682737 7440 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:38.683684 7440 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:38.689872 7441 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:38.690729 7441 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:38.699630 7441 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:38.707643 7304 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:38.707937 7304 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 3.
I20251212 08:53:38.708560 7189 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:38.708832 7407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:38.708899 7441 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 3 FOLLOWER]: Leader pre-election won for term 4
I20251212 08:53:38.708994 7441 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:38.709059 7407 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 3.
I20251212 08:53:38.709120 7441 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 3 FOLLOWER]: Advancing to term 4
I20251212 08:53:38.709534 7266 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:38.709539 7225 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:38.709817 7440 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20251212 08:53:38.709908 7440 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:38.710007 7440 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Advancing to term 4
I20251212 08:53:38.712105 7407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:38.712224 7441 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:38.712335 7407 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 3.
I20251212 08:53:38.712607 7225 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 in current term 4: Already voted for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in this term.
I20251212 08:53:38.712961 7441 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 4 election: Requested vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:38.712956 7440 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:38.713470 7407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:38.713471 7304 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:38.713680 7407 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Advancing to term 4
I20251212 08:53:38.713735 7304 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in current term 4: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:53:38.714051 7440 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 4 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:38.714545 7225 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:38.714545 7406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:38.714846 7225 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 in current term 4: Already voted for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in this term.
I20251212 08:53:38.715209 7265 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed
I20251212 08:53:38.715531 7440 raft_consensus.cc:2749] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20251212 08:53:38.716534 7407 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 4.
I20251212 08:53:38.716919 7188 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:38.717236 7441 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Leader election won for term 4
I20251212 08:53:38.717624 7441 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 LEADER]: Becoming Leader. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Running, Role: LEADER
I20251212 08:53:38.717921 7441 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:38.720664 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed reported cstate change: term changed from 3 to 4, leader changed from 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2) to 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4). New cstate: current_term: 4 leader_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:38.729405 7304 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Refusing update from remote peer 808a87a7594d4fa0b5f8d1c78aebb8ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20251212 08:53:38.729458 7407 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Refusing update from remote peer 808a87a7594d4fa0b5f8d1c78aebb8ed: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 8. (index mismatch)
I20251212 08:53:38.729943 7441 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20251212 08:53:38.730187 7450 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20251212 08:53:38.736758 832 tablet_server.cc:178] TabletServer@127.0.208.3:39569 shutting down...
I20251212 08:53:38.768992 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:38.769312 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:38.769517 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 3 LEADER]: Raft consensus shutting down.
I20251212 08:53:38.769865 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 3 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:38.784327 832 tablet_server.cc:195] TabletServer@127.0.208.3:39569 shutdown complete.
I20251212 08:53:38.787847 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:38.790532 7466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:38.790746 7467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:38.791838 7469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:38.791947 832 server_base.cc:1047] running on GCE node
I20251212 08:53:38.792423 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:38.792497 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:38.792549 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529618792533 us; error 0 us; skew 500 ppm
I20251212 08:53:38.792680 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:38.794070 832 webserver.cc:492] Webserver started at http://127.0.208.3:35191/ using document root <none> and password file <none>
I20251212 08:53:38.794274 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:38.794338 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:38.796195 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:38.797691 7474 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:38.798138 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:38.798236 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:38.798343 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:38.805574 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:38.806053 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:38.806558 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:38.807574 7481 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:38.810338 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:38.810431 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:38.810516 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:38.812556 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:38.812629 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:38.812717 7481 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap starting.
I20251212 08:53:38.823367 7481 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:38.823969 7481 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap complete.
I20251212 08:53:38.824275 7481 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.004s
I20251212 08:53:38.825372 7481 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:38.825649 7481 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Initialized, Role: FOLLOWER
I20251212 08:53:38.826011 7481 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:38.826803 7481 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:38.830188 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39569
I20251212 08:53:38.830229 7539 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39569 every 8 connection(s)
I20251212 08:53:38.838330 832 tablet_server.cc:178] TabletServer@127.0.208.4:45411 shutting down...
I20251212 08:53:38.843793 7540 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:38.844026 7540 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:38.845739 7540 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:38.847633 6482 ts_manager.cc:194] Re-registered known tserver with Master: f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:38.848671 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:38.848956 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:38.849165 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 LEADER]: Raft consensus shutting down.
I20251212 08:53:38.849301 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38274
I20251212 08:53:38.849519 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:38.854398 832 tablet_server.cc:195] TabletServer@127.0.208.4:45411 shutdown complete.
I20251212 08:53:38.857757 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:38.860606 7545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:38.860620 7544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:38.861256 7547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:38.861483 832 server_base.cc:1047] running on GCE node
I20251212 08:53:38.861711 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:38.861773 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:38.861807 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529618861794 us; error 0 us; skew 500 ppm
I20251212 08:53:38.861932 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:38.863185 832 webserver.cc:492] Webserver started at http://127.0.208.4:44337/ using document root <none> and password file <none>
I20251212 08:53:38.863381 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:38.863437 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:38.865264 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:38.866609 7552 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:38.866972 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:38.867107 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:53:38.867231 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:38.890944 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:38.891513 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:38.892113 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:38.895643 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:38.895749 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:38.897814 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:38.897910 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:38.897994 7559 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap starting.
I20251212 08:53:38.908187 7559 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=5 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:38.908818 7559 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap complete.
I20251212 08:53:38.909155 7559 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.004s
I20251212 08:53:38.910000 7559 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:38.910189 7559 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Initialized, Role: FOLLOWER
I20251212 08:53:38.910485 7559 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:38.911343 7559 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20251212 08:53:38.914577 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.4:45411
I20251212 08:53:38.914597 7617 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.4:45411 every 8 connection(s)
I20251212 08:53:38.923043 7618 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
W20251212 08:53:38.923069 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:38.923233 7618 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:38.923646 7618 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:38.924729 6482 ts_manager.cc:194] Re-registered known tserver with Master: 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411)
I20251212 08:53:38.925864 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38278
W20251212 08:53:38.926810 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.933494 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.941861 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:38.951884 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.959828 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.970165 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:38.982124 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:38.992434 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.008248 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.022255 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.036366 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.054055 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.069192 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.086349 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.105036 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.127172 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.148208 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.172124 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.194204 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.218174 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.244691 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.272883 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.299074 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.327991 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.355113 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.385432 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.419227 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.449486 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.484707 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.520272 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.555397 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.592695 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.630499 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.670629 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.717229 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.761859 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:39.803160 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.845319 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:39.850704 7540 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:39.890115 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:39.894402 7623 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:39.894564 7623 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:39.895315 7623 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:39.895707 7407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:39.901825 7514 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:39.902071 7514 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 3.
I20251212 08:53:39.902487 7267 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9
I20251212 08:53:39.902771 7623 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20251212 08:53:39.902881 7623 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:39.902979 7623 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 3 FOLLOWER]: Advancing to term 4
I20251212 08:53:39.905215 7623 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:39.905827 7623 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 4 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:39.906293 7514 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:39.906347 7407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:39.906474 7514 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 3 FOLLOWER]: Advancing to term 4
I20251212 08:53:39.908814 7514 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 4.
I20251212 08:53:39.909227 7267 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9
I20251212 08:53:39.909564 7623 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Leader election won for term 4
I20251212 08:53:39.910029 7623 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:39.910456 7623 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:39.913902 6482 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 3 to 4, leader changed from f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 4 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:39.927291 7618 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:39.933022 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:39.976198 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.024901 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:40.071436 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.117630 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:40.133932 7632 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:40.134145 7632 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:40.134982 7632 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:40.142083 7304 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:40.142410 7514 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:40.142658 7514 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 4: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:53:40.143042 7370 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9; no voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86
I20251212 08:53:40.143322 7632 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 3 FOLLOWER]: Advancing to term 4
I20251212 08:53:40.147166 7632 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20251212 08:53:40.167157 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:40.215652 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.265051 7386 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57638: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:40.312639 7407 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20251212 08:53:40.313691 7623 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20251212 08:53:40.319844 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:40.320792 7514 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 4 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20251212 08:53:40.327785 7626 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20251212 08:53:40.338223 7626 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:53:40.338404 7626 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:40.339054 7626 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:40.339623 7407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:40.340366 7632 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:53:40.340551 7632 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:40.341336 7632 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:40.341964 7304 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:40.342203 7304 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 4.
I20251212 08:53:40.342710 7370 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:40.343408 7632 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20251212 08:53:40.343518 7632 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:53:40.343631 7632 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Advancing to term 5
I20251212 08:53:40.346673 7632 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:40.347438 7632 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 5 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:40.347774 7304 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:40.347975 7304 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Advancing to term 5
I20251212 08:53:40.351424 7304 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 5.
I20251212 08:53:40.351948 7370 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:40.352063 7592 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:40.352267 7632 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Leader election won for term 5
I20251212 08:53:40.352321 7592 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 4.
I20251212 08:53:40.352941 7632 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:53:40.353255 7632 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 4.9, Last appended by leader: 9, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:40.353539 7265 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9
I20251212 08:53:40.354676 7626 raft_consensus.cc:2764] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20251212 08:53:40.355296 7592 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:40.355403 7591 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 9 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:40.355506 7592 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 4.
I20251212 08:53:40.356920 6482 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 4 to 5, leader changed from 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4) to 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2). New cstate: current_term: 5 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } }
W20251212 08:53:40.375419 7283 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34228: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:40.431119 7591 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 4 FOLLOWER]: Advancing to term 5
I20251212 08:53:40.431195 7304 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 9. Preceding OpId from leader: term: 5 index: 11. (index mismatch)
I20251212 08:53:40.431830 7632 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20251212 08:53:40.434033 7591 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 5 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 4 index: 9. Preceding OpId from leader: term: 5 index: 11. (index mismatch)
I20251212 08:53:40.434712 7632 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20251212 08:53:40.437525 832 tablet_server.cc:178] TabletServer@127.0.208.1:35903 shutting down...
I20251212 08:53:40.449065 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:40.449326 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:40.449520 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 LEADER]: Raft consensus shutting down.
I20251212 08:53:40.449824 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:40.450472 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:40.450608 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:40.450788 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:40.466092 832 tablet_server.cc:195] TabletServer@127.0.208.1:35903 shutdown complete.
I20251212 08:53:40.469735 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:40.472373 7657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:40.472386 7656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:40.472959 832 server_base.cc:1047] running on GCE node
W20251212 08:53:40.473217 7659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:40.473527 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:40.473594 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:40.473641 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529620473626 us; error 0 us; skew 500 ppm
I20251212 08:53:40.473783 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:40.474960 832 webserver.cc:492] Webserver started at http://127.0.208.1:36883/ using document root <none> and password file <none>
I20251212 08:53:40.475149 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:40.475205 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:40.477089 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:40.478403 7664 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:40.478766 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:40.478897 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:40.479027 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:40.495980 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:40.496613 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:40.497233 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:40.498422 7671 ts_tablet_manager.cc:548] Loading tablet metadata (0/2 complete)
I20251212 08:53:40.503163 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:40.503274 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20251212 08:53:40.503404 832 ts_tablet_manager.cc:600] Registering tablets (0/2 complete)
I20251212 08:53:40.505467 7671 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:40.507102 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:40.507186 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20251212 08:53:40.516358 7671 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=6 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:40.516810 7671 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:40.517086 7671 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20251212 08:53:40.517733 7671 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:40.517922 7671 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:40.518266 7671 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 5.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:40.519075 7671 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:40.519333 7671 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:40.524382 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35903
I20251212 08:53:40.524412 7730 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35903 every 8 connection(s)
I20251212 08:53:40.529042 7671 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:40.529721 7671 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:40.530089 7671 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.011s user 0.010s sys 0.000s
I20251212 08:53:40.531236 7671 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:40.531580 7671 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:40.531944 7671 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:40.532622 7671 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:53:40.537966 832 tablet_server.cc:178] TabletServer@127.0.208.2:42203 shutting down...
I20251212 08:53:40.551203 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:40.551720 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:40.551915 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:40.552088 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:40.552969 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:40.553135 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 LEADER]: Raft consensus shutting down.
I20251212 08:53:40.553488 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:40.558931 7731 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:40.559113 7731 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:40.559218 832 tablet_server.cc:195] TabletServer@127.0.208.2:42203 shutdown complete.
I20251212 08:53:40.559592 7731 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:40.560885 6482 ts_manager.cc:194] Re-registered known tserver with Master: 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:40.562194 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38290
I20251212 08:53:40.563205 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251212 08:53:40.563483 7731 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:40.565984 7739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:40.566027 7740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:40.566587 7742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:40.567894 832 server_base.cc:1047] running on GCE node
I20251212 08:53:40.568121 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:40.568186 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:40.568221 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529620568208 us; error 0 us; skew 500 ppm
I20251212 08:53:40.568379 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:40.569533 832 webserver.cc:492] Webserver started at http://127.0.208.2:36849/ using document root <none> and password file <none>
I20251212 08:53:40.569789 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:40.569855 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:40.571846 832 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:40.573401 7747 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:40.573843 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:40.573969 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:40.574105 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:40.589716 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:40.590263 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:40.590819 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:40.596545 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:40.596653 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20251212 08:53:40.598744 7754 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:40.600854 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:40.600960 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.004s sys 0.000s
I20251212 08:53:40.610795 7754 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=12 overwritten=0 applied=12 ignored=0} inserts{seen=7 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:40.611296 7754 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:40.611614 7754 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.003s
I20251212 08:53:40.612383 7754 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:40.612633 7754 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:40.613011 7754 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 12, Last appended: 5.12, Last appended by leader: 12, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:40.613907 7754 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20251212 08:53:40.614271 7754 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:40.619592 7813 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:42203 every 8 connection(s)
I20251212 08:53:40.619601 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:42203
I20251212 08:53:40.622664 7754 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:40.623292 7754 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:40.623598 7754 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20251212 08:53:40.624459 7754 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:40.624670 7754 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:40.624917 7754 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:40.625420 7754 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:40.628305 7814 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
W20251212 08:53:40.628365 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:40.628476 7814 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:40.628908 7814 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:40.630089 6482 ts_manager.cc:194] Re-registered known tserver with Master: 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:40.631791 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38302
I20251212 08:53:40.632943 7814 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:40.636694 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.645671 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.652311 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.661254 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.671181 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.682555 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.694706 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.706790 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.720404 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.732550 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.745573 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.764250 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.779492 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.795768 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.814460 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.834494 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.856917 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.881639 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.904863 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.930017 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.954676 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:40.980871 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.008035 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.037734 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.065706 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.097836 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.130752 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.161885 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.197310 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.231570 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.267796 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.306167 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.345599 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.384764 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.421756 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.464447 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.505666 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.545683 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.589901 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.637027 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.683015 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.729573 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.776885 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:41.798913 7819 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:41.799108 7819 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:41.799963 7819 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:41.819197 7704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:41.819226 7591 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:41.819844 7749 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9; no voters: 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:41.820101 7819 raft_consensus.cc:2749] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20251212 08:53:41.824692 7494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43896: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:41.830672 7824 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:41.830840 7824 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:41.831532 7824 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:41.839296 7704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:41.839732 7704 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 4.
I20251212 08:53:41.841305 7478 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:41.841435 7819 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:41.841673 7824 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20251212 08:53:41.841661 7819 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:41.841859 7824 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:41.841926 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:41.842069 7824 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 4 FOLLOWER]: Advancing to term 5
I20251212 08:53:41.842212 7787 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 4.
I20251212 08:53:41.842541 7819 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:41.843855 7704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:41.844071 7704 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 4.
I20251212 08:53:41.844441 7751 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:41.844748 7819 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20251212 08:53:41.844872 7819 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:41.844897 7824 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:41.844990 7819 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 4 FOLLOWER]: Advancing to term 5
I20251212 08:53:41.845577 7824 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 5 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:41.846026 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:41.846136 7704 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:41.846374 7704 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 4 FOLLOWER]: Advancing to term 5
I20251212 08:53:41.848093 7819 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:41.848655 7819 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 5 election: Requested vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:41.848906 7787 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f2ceec6428fb4ee0b98c750a718d3a86 in current term 5: Already voted for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in this term.
I20251212 08:53:41.849141 7704 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 5.
I20251212 08:53:41.849454 7703 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:41.849644 7478 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9
I20251212 08:53:41.849682 7703 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 5: Already voted for candidate f2ceec6428fb4ee0b98c750a718d3a86 in this term.
I20251212 08:53:41.849808 7514 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:41.849998 7514 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 5: Already voted for candidate f2ceec6428fb4ee0b98c750a718d3a86 in this term.
I20251212 08:53:41.850114 7824 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 FOLLOWER]: Leader election won for term 5
I20251212 08:53:41.850283 7513 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:41.850586 7824 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 LEADER]: Becoming Leader. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Running, Role: LEADER
I20251212 08:53:41.850862 7824 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:41.852391 7751 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9; no voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86
I20251212 08:53:41.852764 7819 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20251212 08:53:41.853411 6482 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 reported cstate change: term changed from 4 to 5, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3). New cstate: current_term: 5 leader_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:41.876829 7787 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 9. (index mismatch)
I20251212 08:53:41.876830 7703 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 9. (index mismatch)
I20251212 08:53:41.877416 7824 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20251212 08:53:41.877594 7829 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20251212 08:53:41.886886 832 tablet_server.cc:178] TabletServer@127.0.208.3:39569 shutting down...
I20251212 08:53:41.895143 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:41.895404 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:41.895609 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 LEADER]: Raft consensus shutting down.
I20251212 08:53:41.895939 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:41.901629 832 tablet_server.cc:195] TabletServer@127.0.208.3:39569 shutdown complete.
I20251212 08:53:41.905059 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:41.907661 7846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:41.907714 7845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:41.908654 832 server_base.cc:1047] running on GCE node
W20251212 08:53:41.908638 7848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:41.909096 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:41.909165 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:41.909224 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529621909202 us; error 0 us; skew 500 ppm
I20251212 08:53:41.909356 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:41.910555 832 webserver.cc:492] Webserver started at http://127.0.208.3:35191/ using document root <none> and password file <none>
I20251212 08:53:41.910753 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:41.910813 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:41.912755 832 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:41.914088 7853 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:41.914515 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:41.914623 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:41.914757 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:41.924875 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:41.925369 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:41.925891 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:41.926996 7860 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:41.929797 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:41.929903 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:41.930032 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:41.932065 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:41.932149 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20251212 08:53:41.932267 7860 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap starting.
I20251212 08:53:41.942588 7860 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=9 overwritten=0 applied=9 ignored=0} inserts{seen=4 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:41.943077 7860 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap complete.
I20251212 08:53:41.943410 7860 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20251212 08:53:41.944090 7860 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:41.944316 7860 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Initialized, Role: FOLLOWER
I20251212 08:53:41.944631 7860 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 5.9, Last appended by leader: 9, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:41.945366 7860 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent starting tablet: real 0.002s user 0.002s sys 0.003s
I20251212 08:53:41.948171 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39569
I20251212 08:53:41.948195 7918 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39569 every 8 connection(s)
W20251212 08:53:41.950698 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.953336 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:41.955323 7919 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:41.955464 7919 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:41.955803 7919 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:41.956840 6482 ts_manager.cc:194] Re-registered known tserver with Master: f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:41.957921 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38318
W20251212 08:53:41.959769 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.968950 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.978847 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:41.985558 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:41.995908 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.004881 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.016014 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.031512 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:42.037971 7923 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:42.038190 7923 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:42.038995 7923 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:42.044799 7703 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:42.044838 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:42.045064 7787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 5.
I20251212 08:53:42.045393 7555 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:42.045683 7923 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 5 FOLLOWER]: Leader pre-election won for term 6
I20251212 08:53:42.045783 7923 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 5 FOLLOWER]: Starting leader election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:42.045861 7923 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 5 FOLLOWER]: Advancing to term 6
W20251212 08:53:42.047288 7572 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60646: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:42.048224 7923 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:42.048815 7923 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 6 election: Requested vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:42.049206 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:42.049366 7787 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Advancing to term 6
I20251212 08:53:42.050675 7703 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 6 candidate_status { last_received { term: 5 index: 12 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:42.050876 7703 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Advancing to term 6
I20251212 08:53:42.052102 7787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 6.
I20251212 08:53:42.052572 7555 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters:
I20251212 08:53:42.052853 7923 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Leader election won for term 6
I20251212 08:53:42.053256 7923 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 LEADER]: Becoming Leader. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Running, Role: LEADER
I20251212 08:53:42.053550 7923 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 5.12, Last appended by leader: 12, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:42.053654 7703 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 6.
I20251212 08:53:42.055830 6482 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed reported cstate change: term changed from 5 to 6, leader changed from 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2) to 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4). New cstate: current_term: 6 leader_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } }
W20251212 08:53:42.062744 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:42.082805 7787 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Refusing update from remote peer 808a87a7594d4fa0b5f8d1c78aebb8ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 12. Preceding OpId from leader: term: 6 index: 14. (index mismatch)
I20251212 08:53:42.082810 7703 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Refusing update from remote peer 808a87a7594d4fa0b5f8d1c78aebb8ed: Log matching property violated. Preceding OpId in replica: term: 5 index: 12. Preceding OpId from leader: term: 6 index: 14. (index mismatch)
I20251212 08:53:42.083408 7926 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20251212 08:53:42.083676 7923 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20251212 08:53:42.091331 832 tablet_server.cc:178] TabletServer@127.0.208.4:45411 shutting down...
I20251212 08:53:42.099609 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:42.099864 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:42.100035 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 LEADER]: Raft consensus shutting down.
I20251212 08:53:42.100399 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:42.114883 832 tablet_server.cc:195] TabletServer@127.0.208.4:45411 shutdown complete.
I20251212 08:53:42.118470 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:42.121178 7933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:42.121402 7934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:42.121670 7936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:42.121744 832 server_base.cc:1047] running on GCE node
I20251212 08:53:42.121996 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:42.122061 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:42.122114 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529622122096 us; error 0 us; skew 500 ppm
I20251212 08:53:42.122264 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:42.123392 832 webserver.cc:492] Webserver started at http://127.0.208.4:44337/ using document root <none> and password file <none>
I20251212 08:53:42.123596 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:42.123669 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:42.125445 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:42.126585 7941 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:42.126907 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:42.127015 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:53:42.127138 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:42.138067 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:42.138557 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:42.139124 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:42.142710 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:42.142814 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:42.144898 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:42.144989 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:42.145120 7948 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap starting.
I20251212 08:53:42.159480 7948 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:42.159965 7948 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap complete.
I20251212 08:53:42.160315 7948 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent bootstrapping tablet: real 0.015s user 0.011s sys 0.000s
I20251212 08:53:42.161031 7948 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:42.161206 7948 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Initialized, Role: FOLLOWER
I20251212 08:53:42.161439 7948 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 6.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:42.162146 7948 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:53:42.162746 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.4:45411
I20251212 08:53:42.162804 8006 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.4:45411 every 8 connection(s)
I20251212 08:53:42.169695 8007 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:42.169843 8007 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:42.170149 8007 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:42.171231 6480 ts_manager.cc:194] Re-registered known tserver with Master: 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411)
W20251212 08:53:42.171419 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:42.172479 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:38332
W20251212 08:53:42.174015 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.179738 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.190160 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.195868 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.202806 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.212141 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.223135 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.236089 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.250852 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.264842 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.280068 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.296811 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.314888 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.334884 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.357281 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.379398 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.399542 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.425148 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.446336 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.470577 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.499557 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.524684 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.551744 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.581959 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.610083 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.639384 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.672191 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.702373 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.742993 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.778762 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.815896 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.852144 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:42.892897 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:42.931864 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:42.959295 7919 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:42.969192 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.010064 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:43.050170 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.094357 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.137210 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:43.173759 8007 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:43.182430 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.226712 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.274729 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:43.321987 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.370203 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:43.383987 8012 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:43.384316 8012 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:43.385042 8013 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:43.385176 8012 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:43.385272 8013 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:43.387190 8013 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:43.387616 7703 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:43.387924 7703 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 5.
I20251212 08:53:43.389212 7751 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:43.389659 8013 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20251212 08:53:43.389783 8013 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:43.389917 8013 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 5 FOLLOWER]: Advancing to term 6
I20251212 08:53:43.393723 8013 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:43.393851 7893 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:43.394121 7893 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 5.
I20251212 08:53:43.394615 7668 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:43.394901 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:43.394925 8012 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20251212 08:53:43.395175 7787 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 in current term 6: Already voted for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in this term.
I20251212 08:53:43.395174 8013 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 6 election: Requested vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:43.395324 8012 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:43.395717 8012 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 5 FOLLOWER]: Advancing to term 6
I20251212 08:53:43.395330 7703 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:43.395888 7893 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:43.396008 7892 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:43.396116 7893 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 5.
I20251212 08:53:43.398205 8012 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:43.398545 7703 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 6: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:53:43.398798 8012 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 6 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:43.399114 7751 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9; no voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86
I20251212 08:53:43.399223 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:43.399390 8013 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20251212 08:53:43.399223 7892 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 6 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:43.399607 7892 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 5 FOLLOWER]: Advancing to term 6
I20251212 08:53:43.400468 7787 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 in current term 6: Already voted for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in this term.
I20251212 08:53:43.401979 7892 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 6.
I20251212 08:53:43.402379 7668 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9
I20251212 08:53:43.402693 8012 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Leader election won for term 6
I20251212 08:53:43.403090 8012 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:43.403339 8012 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.9, Last appended by leader: 9, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:43.405952 6480 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 5 to 6, leader changed from f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 6 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: UNKNOWN } } }
W20251212 08:53:43.422710 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:43.472990 7684 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34282: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.523222 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:43.551254 8026 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:43.551523 8026 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:43.552371 8026 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:43.558542 7703 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:43.558542 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:43.559192 7944 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:43.559480 8026 raft_consensus.cc:2749] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20251212 08:53:43.575069 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:43.587697 8012 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:53:43.587914 8012 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:43.588712 8012 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:43.589262 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:43.589548 7787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 6.
I20251212 08:53:43.589712 8013 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:53:43.589905 8013 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:43.590037 7667 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:43.590328 8012 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20251212 08:53:43.590430 8012 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:53:43.590545 8012 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Advancing to term 7
I20251212 08:53:43.590593 8013 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:43.591137 7703 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:43.593906 8012 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:43.594352 7703 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 7: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:53:43.595098 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:43.595331 7787 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Advancing to term 7
I20251212 08:53:43.595410 7981 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:43.595499 7980 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:43.595582 8012 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 7 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:43.595618 7981 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 6.
I20251212 08:53:43.598021 7981 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 7 candidate_status { last_received { term: 6 index: 14 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:43.598253 7981 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 6.
I20251212 08:53:43.598590 7787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 7.
I20251212 08:53:43.598645 7749 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:43.598914 8013 raft_consensus.cc:2764] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20251212 08:53:43.598999 7667 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters: 808a87a7594d4fa0b5f8d1c78aebb8ed
I20251212 08:53:43.599247 8012 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Leader election won for term 7
I20251212 08:53:43.599416 8012 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:43.599707 8012 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 6.14, Last appended by leader: 14, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:43.602319 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 6 to 7, leader changed from 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 7 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:43.628237 7787 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 6 index: 14. Preceding OpId from leader: term: 7 index: 16. (index mismatch)
I20251212 08:53:43.628170 7981 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 6 FOLLOWER]: Advancing to term 7
I20251212 08:53:43.628842 8012 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20251212 08:53:43.631249 7981 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 7 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 6 index: 14. Preceding OpId from leader: term: 7 index: 16. (index mismatch)
I20251212 08:53:43.631884 8012 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20251212 08:53:43.634826 832 tablet_server.cc:178] TabletServer@127.0.208.1:35903 shutting down...
I20251212 08:53:43.650085 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:43.650437 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:43.650689 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 LEADER]: Raft consensus shutting down.
I20251212 08:53:43.650974 832 pending_rounds.cc:70] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Trying to abort 1 pending ops.
I20251212 08:53:43.651046 832 pending_rounds.cc:77] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Aborting op as it isn't in flight: id { term: 6 index: 10 } timestamp: 7231609337462341632 op_type: NO_OP noop_request { }
I20251212 08:53:43.651172 832 raft_consensus.cc:2889] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251212 08:53:43.651286 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:43.652037 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:43.652192 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 LEADER]: Raft consensus shutting down.
I20251212 08:53:43.656734 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:43.672214 832 tablet_server.cc:195] TabletServer@127.0.208.1:35903 shutdown complete.
I20251212 08:53:43.676396 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:43.679000 8042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:43.679241 8043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:43.679590 832 server_base.cc:1047] running on GCE node
W20251212 08:53:43.679694 8045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:43.680079 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:43.680146 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:43.680207 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529623680188 us; error 0 us; skew 500 ppm
I20251212 08:53:43.680411 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:43.681699 832 webserver.cc:492] Webserver started at http://127.0.208.1:36883/ using document root <none> and password file <none>
I20251212 08:53:43.681933 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:43.682000 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:43.684027 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:43.685451 8050 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:43.685849 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:43.685974 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:43.686103 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:43.699337 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:43.699890 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:43.700563 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:43.701728 8057 ts_tablet_manager.cc:548] Loading tablet metadata (0/2 complete)
I20251212 08:53:43.706466 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:43.706586 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20251212 08:53:43.706698 832 ts_tablet_manager.cc:600] Registering tablets (0/2 complete)
I20251212 08:53:43.708994 8057 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:43.710906 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:43.711019 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:43.724222 8057 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=9 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:43.724725 8057 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:43.725000 8057 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.004s
I20251212 08:53:43.725817 8057 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:43.726008 8057 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:43.726315 8057 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 7.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:43.727039 8057 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:43.727293 8057 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:43.728688 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35903
I20251212 08:53:43.728688 8115 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35903 every 8 connection(s)
W20251212 08:53:43.736608 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.741536 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:43.741873 8057 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=9 ignored=0} inserts{seen=4 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251212 08:53:43.742425 8057 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:43.742708 8057 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.015s user 0.014s sys 0.000s
I20251212 08:53:43.743515 8057 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:43.743752 8116 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:43.743781 8057 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:43.743935 8116 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:43.744071 8057 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 9, Last appended: 6.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:43.744338 8116 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:43.744560 8057 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:43.745469 6482 ts_manager.cc:194] Re-registered known tserver with Master: 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:43.747229 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33362
I20251212 08:53:43.748518 8116 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:43.751914 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.759344 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.767344 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.774653 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.785655 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.797813 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.810019 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.826568 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.842696 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.857846 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.877656 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.896849 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.913053 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.935593 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.954689 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:43.975946 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.000734 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.026019 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.049248 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.073937 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.098929 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.127919 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.155431 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.186702 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.218731 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.249451 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.283696 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.315809 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.350445 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.386698 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.424872 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.465708 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.501995 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.541088 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.581668 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.623732 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.663918 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.710772 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.756748 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.801047 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.849840 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.899029 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.945111 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:44.996748 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.044855 7767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:57678: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:45.095649 8123 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:45.095882 8123 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
W20251212 08:53:45.096328 7873 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45180: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:45.096827 8123 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:45.110687 8128 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:45.110934 8128 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:45.111974 8128 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:45.119596 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 7 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:45.119916 7787 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 6.
I20251212 08:53:45.120445 7856 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:45.120793 8123 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20251212 08:53:45.120923 8123 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:45.121053 8123 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 6 FOLLOWER]: Advancing to term 7
I20251212 08:53:45.125185 8123 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:45.125826 8090 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 7 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:45.126207 8090 raft_consensus.cc:2410] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2ceec6428fb4ee0b98c750a718d3a86 for term 7 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20251212 08:53:45.126457 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 7 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:45.126591 8089 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 7 candidate_status { last_received { term: 5 index: 9 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:45.125923 8123 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 7 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:45.126791 8089 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 6 FOLLOWER]: Advancing to term 7
I20251212 08:53:45.126648 7787 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 6 FOLLOWER]: Advancing to term 7
I20251212 08:53:45.129676 8089 raft_consensus.cc:2410] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f2ceec6428fb4ee0b98c750a718d3a86 for term 7 because replica has last-logged OpId of term: 6 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 9.
I20251212 08:53:45.130358 7787 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 7.
I20251212 08:53:45.130702 7856 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters: 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:45.130987 8123 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Leader election won for term 7
I20251212 08:53:45.131640 8123 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 LEADER]: Becoming Leader. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Running, Role: LEADER
I20251212 08:53:45.132004 8123 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 5.9, Last appended by leader: 9, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:45.132134 7892 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 7 candidate_status { last_received { term: 6 index: 10 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:45.133092 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 7 candidate_status { last_received { term: 6 index: 10 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:45.133343 7787 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 in current term 7: Already voted for candidate f2ceec6428fb4ee0b98c750a718d3a86 in this term.
I20251212 08:53:45.133705 8053 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86
I20251212 08:53:45.133956 8128 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20251212 08:53:45.134513 8135 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:45.134645 8135 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:45.134696 6482 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 reported cstate change: term changed from 6 to 7, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3). New cstate: current_term: 7 leader_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:45.135262 8135 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:45.135548 7981 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:45.141142 8089 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:45.141366 8089 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 7.
I20251212 08:53:45.141690 7751 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters: 808a87a7594d4fa0b5f8d1c78aebb8ed
I20251212 08:53:45.141976 8135 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20251212 08:53:45.142072 8135 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:45.142168 8135 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Advancing to term 8
I20251212 08:53:45.144075 8135 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:45.144619 8135 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 8 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:45.145053 8089 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:45.145053 7981 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:45.145303 8089 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Advancing to term 8
I20251212 08:53:45.146490 8141 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:45.146641 8141 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:45.147187 8141 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:45.147624 7787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:45.147842 7787 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in current term 8: Already voted for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in this term.
I20251212 08:53:45.147876 8089 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 8.
I20251212 08:53:45.148231 7751 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters: 808a87a7594d4fa0b5f8d1c78aebb8ed
I20251212 08:53:45.148545 8135 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Leader election won for term 8
I20251212 08:53:45.148983 8135 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:53:45.149261 8135 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 7.16, Last appended by leader: 16, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:45.151863 6482 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 7 to 8, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2). New cstate: current_term: 8 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:45.153918 8089 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 8 candidate_status { last_received { term: 7 index: 16 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:45.154156 8089 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in current term 8: Already voted for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in this term.
I20251212 08:53:45.155089 7945 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:45.155745 8141 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 7 FOLLOWER]: Advancing to term 8
I20251212 08:53:45.158404 8089 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 6 index: 10. Preceding OpId from leader: term: 7 index: 11. (index mismatch)
I20251212 08:53:45.158509 7787 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 5 index: 9. Preceding OpId from leader: term: 7 index: 11. (index mismatch)
I20251212 08:53:45.158521 8141 raft_consensus.cc:2749] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20251212 08:53:45.159040 8123 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20251212 08:53:45.159544 8133 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20251212 08:53:45.159734 8133 consensus_queue.cc:1243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Peer 8a42a66698e144409d7216b8f7b78219 log is divergent from this leader: its last log entry 6.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20251212 08:53:45.160837 8089 pending_rounds.cc:85] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Aborting all ops after (but not including) 9
I20251212 08:53:45.160965 8089 pending_rounds.cc:107] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Aborting uncommitted NO_OP operation due to leader change: 6.10
I20251212 08:53:45.161144 8089 raft_consensus.cc:2889] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251212 08:53:45.171867 832 tablet_server.cc:178] TabletServer@127.0.208.2:42203 shutting down...
I20251212 08:53:45.181440 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:45.181703 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:45.181890 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:45.182122 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:45.182852 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:45.182971 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 LEADER]: Raft consensus shutting down.
I20251212 08:53:45.183189 832 pending_rounds.cc:70] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Trying to abort 1 pending ops.
I20251212 08:53:45.183243 832 pending_rounds.cc:77] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Aborting op as it isn't in flight: id { term: 8 index: 17 } timestamp: 7231609344614031360 op_type: NO_OP noop_request { }
I20251212 08:53:45.183333 832 raft_consensus.cc:2889] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251212 08:53:45.183411 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:45.198192 832 tablet_server.cc:195] TabletServer@127.0.208.2:42203 shutdown complete.
I20251212 08:53:45.201978 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:45.204413 8153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:45.204681 8154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:45.205322 8156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:45.205808 832 server_base.cc:1047] running on GCE node
I20251212 08:53:45.206121 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:45.206187 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:45.206279 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529625206259 us; error 0 us; skew 500 ppm
I20251212 08:53:45.206442 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:45.207705 832 webserver.cc:492] Webserver started at http://127.0.208.2:36849/ using document root <none> and password file <none>
I20251212 08:53:45.207922 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:45.207989 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:45.209985 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:45.211490 8161 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:45.211870 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:45.211997 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:45.212126 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:45.221387 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:45.221971 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:45.222563 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:45.223656 8168 ts_tablet_manager.cc:548] Loading tablet metadata (0/2 complete)
I20251212 08:53:45.228940 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:45.229066 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20251212 08:53:45.229195 832 ts_tablet_manager.cc:600] Registering tablets (0/2 complete)
I20251212 08:53:45.231333 8168 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:45.233106 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:45.233193 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:45.244704 8168 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=16 ignored=0} inserts{seen=9 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251212 08:53:45.245157 8168 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:45.245429 8168 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.003s
I20251212 08:53:45.246021 8168 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:45.246245 8168 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:45.246506 8168 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 8.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:45.247208 8168 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:45.247453 8168 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:45.257629 8168 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=5 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:45.258085 8168 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:45.258394 8168 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20251212 08:53:45.258934 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:42203
I20251212 08:53:45.258984 8227 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:42203 every 8 connection(s)
I20251212 08:53:45.259178 8168 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:45.259317 8168 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:45.259517 8168 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 7.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:45.260136 8168 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20251212 08:53:45.261955 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.264448 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:45.265848 8228 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:45.265959 8228 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:45.266268 8228 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:45.267338 6480 ts_manager.cc:194] Re-registered known tserver with Master: 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:45.268630 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33374
I20251212 08:53:45.269723 8228 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:45.275823 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.285176 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.291994 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.300033 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.312551 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.323851 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.333891 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.346288 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.359138 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.373286 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.388756 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.406772 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.422786 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.441371 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.463479 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.482820 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.508351 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.531473 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.557724 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.585806 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.613988 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.643002 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.674424 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.705474 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.736511 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.768003 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.799274 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.833364 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.871019 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.907179 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:45.944370 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:45.985270 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.022330 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:46.060513 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.100064 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.140126 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:46.181233 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.225979 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.270259 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:46.315461 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.365072 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.412479 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:46.462162 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.512007 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.562096 7961 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:60680: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:46.601784 8235 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:46.602021 8235 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:46.602919 8235 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
W20251212 08:53:46.615551 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:46.619102 7981 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:46.619161 8089 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:46.619357 8089 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 8.
I20251212 08:53:46.619362 7981 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 8.
I20251212 08:53:46.619792 8165 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:46.620061 8235 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20251212 08:53:46.620148 8235 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:46.620215 8235 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Advancing to term 9
I20251212 08:53:46.622310 8235 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:46.622900 8235 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 9 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:46.623360 7981 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:46.623394 8089 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 17 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:46.623533 7981 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 8 FOLLOWER]: Advancing to term 9
I20251212 08:53:46.623554 8089 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Advancing to term 9
I20251212 08:53:46.625730 8089 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 9.
I20251212 08:53:46.625726 7981 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 9.
I20251212 08:53:46.626154 8163 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters:
I20251212 08:53:46.626442 8235 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Leader election won for term 9
I20251212 08:53:46.626842 8235 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:53:46.627100 8235 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 8.17, Last appended by leader: 17, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:46.629630 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 8 to 9. New cstate: current_term: 9 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:46.671504 7981 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 16. Preceding OpId from leader: term: 9 index: 19. (index mismatch)
I20251212 08:53:46.671525 8089 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 7 index: 16. Preceding OpId from leader: term: 9 index: 19. (index mismatch)
I20251212 08:53:46.672093 8240 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20251212 08:53:46.672426 8235 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20251212 08:53:46.677021 8246 mvcc.cc:204] Tried to move back new op lower bound from 7231609350840549376 to 7231609350666809344. Current Snapshot: MvccSnapshot[applied={T|T < 7231609350840549376}]
I20251212 08:53:46.682524 832 tablet_server.cc:178] TabletServer@127.0.208.3:39569 shutting down...
I20251212 08:53:46.691782 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:46.692111 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:46.692342 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 LEADER]: Raft consensus shutting down.
I20251212 08:53:46.692653 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:46.707494 832 tablet_server.cc:195] TabletServer@127.0.208.3:39569 shutdown complete.
I20251212 08:53:46.711200 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:46.713939 8257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:46.714123 8258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:46.714401 8260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:46.715131 832 server_base.cc:1047] running on GCE node
I20251212 08:53:46.715400 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:46.715476 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:46.715543 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529626715522 us; error 0 us; skew 500 ppm
I20251212 08:53:46.715708 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:46.717020 832 webserver.cc:492] Webserver started at http://127.0.208.3:35191/ using document root <none> and password file <none>
I20251212 08:53:46.717217 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:46.717275 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:46.719026 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:46.720232 8265 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:46.720609 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:46.720722 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:46.720857 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:46.738514 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:46.739076 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:46.739660 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:46.740803 8272 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:46.743559 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:46.743665 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:46.743808 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:46.745783 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:46.745898 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:46.745978 8272 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap starting.
I20251212 08:53:46.761389 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39569
I20251212 08:53:46.761407 8329 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39569 every 8 connection(s)
I20251212 08:53:46.761901 8272 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=11 overwritten=0 applied=11 ignored=0} inserts{seen=5 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:46.762486 8272 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap complete.
I20251212 08:53:46.762862 8272 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent bootstrapping tablet: real 0.017s user 0.016s sys 0.000s
I20251212 08:53:46.763887 8272 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:46.764232 8272 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Initialized, Role: FOLLOWER
I20251212 08:53:46.764626 8272 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 11, Last appended: 7.11, Last appended by leader: 11, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:46.765468 8272 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:46.770838 8330 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
W20251212 08:53:46.770977 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:46.771027 8330 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:46.771481 8330 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:46.772660 6480 ts_manager.cc:194] Re-registered known tserver with Master: f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:46.773828 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33376
I20251212 08:53:46.775014 8330 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:46.775796 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.781963 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.790625 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.799515 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.808570 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.819801 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.828729 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.841881 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.856508 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.869613 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.885813 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.902670 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.920876 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.938071 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.960676 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:46.978899 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.001292 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.023100 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.047207 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.069478 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.094264 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.121995 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.151203 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.182225 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.209640 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.239825 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.274750 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.308885 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.340484 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.375312 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.410326 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.444684 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.481544 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.518575 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.558692 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.602576 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.641834 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.684921 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.730507 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.775799 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.820966 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.869640 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.915935 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:47.962373 8070 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42312: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:47.974227 8337 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:47.974470 8337 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:47.975332 8337 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:47.989156 8203 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 8 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:47.989210 8089 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 8 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:47.989813 8268 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2ceec6428fb4ee0b98c750a718d3a86; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:47.990123 8337 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20251212 08:53:48.011294 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.059259 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:48.089653 8342 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:48.089876 8342 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:48.090617 8342 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:48.097285 8203 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 8 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:48.097659 8305 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 8 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:48.097865 8305 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 7.
I20251212 08:53:48.098261 8054 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9
I20251212 08:53:48.098547 8342 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20251212 08:53:48.098671 8342 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:48.098767 8342 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 7 FOLLOWER]: Advancing to term 8
I20251212 08:53:48.100997 8342 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:48.101635 8342 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 8 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:48.102018 8203 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 8 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:48.102109 8305 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 8 candidate_status { last_received { term: 7 index: 11 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:48.102241 8305 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 7 FOLLOWER]: Advancing to term 8
I20251212 08:53:48.102245 8203 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 7 FOLLOWER]: Advancing to term 8
I20251212 08:53:48.105352 8203 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 8.
I20251212 08:53:48.105340 8305 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 8.
I20251212 08:53:48.105873 8054 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:48.106248 8342 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Leader election won for term 8
I20251212 08:53:48.106643 8342 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:48.106949 8342 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 7.11, Last appended by leader: 11, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:48.109277 6480 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 7 to 8, leader changed from f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 8 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:48.114133 8203 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 7 index: 11. Preceding OpId from leader: term: 8 index: 13. (index mismatch)
I20251212 08:53:48.114133 8305 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 8 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 7 index: 11. Preceding OpId from leader: term: 8 index: 13. (index mismatch)
I20251212 08:53:48.114817 8346 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20251212 08:53:48.115038 8342 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20251212 08:53:48.124543 832 tablet_server.cc:178] TabletServer@127.0.208.4:45411 shutting down...
I20251212 08:53:48.134115 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:48.134418 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:48.134594 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:48.134850 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:48.148815 832 tablet_server.cc:195] TabletServer@127.0.208.4:45411 shutdown complete.
I20251212 08:53:48.152179 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:48.154954 8361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:48.155261 8362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:48.155371 8364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:48.155812 832 server_base.cc:1047] running on GCE node
I20251212 08:53:48.156114 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:48.156189 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:48.156270 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529628156236 us; error 0 us; skew 500 ppm
I20251212 08:53:48.156618 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:48.158201 832 webserver.cc:492] Webserver started at http://127.0.208.4:44337/ using document root <none> and password file <none>
I20251212 08:53:48.158461 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:48.158556 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:48.160463 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:48.161939 8369 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:48.162525 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:48.162671 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:53:48.162828 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:48.169572 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:48.170154 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:48.170764 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:48.171892 8376 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:48.174597 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:48.174691 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:48.174829 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:48.176879 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:48.176988 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:48.177105 8376 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap starting.
I20251212 08:53:48.195171 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.4:45411
I20251212 08:53:48.195210 8433 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.4:45411 every 8 connection(s)
I20251212 08:53:48.196156 8376 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=10 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:48.196786 8376 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap complete.
I20251212 08:53:48.197245 8376 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.009s
I20251212 08:53:48.198524 8376 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:48.200024 8376 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Initialized, Role: FOLLOWER
I20251212 08:53:48.200522 8376 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 9.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:48.205269 832 tablet_server.cc:178] TabletServer@127.0.208.1:35903 shutting down...
I20251212 08:53:48.216724 8376 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent starting tablet: real 0.019s user 0.011s sys 0.006s
I20251212 08:53:48.233168 8434 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:48.233366 8434 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:48.233776 8434 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:48.235021 6480 ts_manager.cc:194] Re-registered known tserver with Master: 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411)
I20251212 08:53:48.236244 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33392
I20251212 08:53:48.237753 8434 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:48.239327 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:48.239549 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:48.239735 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 LEADER]: Raft consensus shutting down.
I20251212 08:53:48.240051 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:48.240869 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:48.240983 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:48.241132 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:48.245858 832 tablet_server.cc:195] TabletServer@127.0.208.1:35903 shutdown complete.
I20251212 08:53:48.249394 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:48.252151 8439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:48.252187 8440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:48.252595 8442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:48.252889 832 server_base.cc:1047] running on GCE node
I20251212 08:53:48.253217 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:48.253286 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:48.253337 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529628253333 us; error 0 us; skew 500 ppm
I20251212 08:53:48.253479 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:48.254719 832 webserver.cc:492] Webserver started at http://127.0.208.1:36883/ using document root <none> and password file <none>
I20251212 08:53:48.254925 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:48.254983 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:48.256767 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:48.258169 8447 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:48.258502 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:48.258611 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:48.258715 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:48.276268 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:48.276854 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:48.277452 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:48.283267 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:48.283357 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20251212 08:53:48.285550 8454 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:48.287171 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:48.287256 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20251212 08:53:48.300489 8454 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=10 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:48.301028 8454 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:48.301295 8454 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20251212 08:53:48.301962 8454 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:48.302120 8454 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:48.302417 8454 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 9.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:48.303123 8454 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:48.303396 8454 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:48.304780 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35903
I20251212 08:53:48.304828 8512 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35903 every 8 connection(s)
I20251212 08:53:48.314962 8513 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:48.315150 8513 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:48.315480 8513 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:48.316716 6480 ts_manager.cc:194] Re-registered known tserver with Master: 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:48.318516 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33394
W20251212 08:53:48.318614 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:48.320119 8454 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=1 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:48.320544 8454 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:48.320760 8454 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.009s
I20251212 08:53:48.321353 8454 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:48.321514 8454 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:48.321733 8454 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 8.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:48.322183 8513 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:48.322211 8454 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
W20251212 08:53:48.326283 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.332342 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.341243 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.347998 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.361444 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.374934 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.388068 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.404750 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.417670 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.434686 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.452387 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.471508 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.487576 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.509148 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.527101 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.549314 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.574056 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.597476 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.623405 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.651237 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.679423 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.708451 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.736897 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.767940 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.800028 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.832363 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.862396 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.894505 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.928079 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.965315 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:48.999537 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.037390 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.074528 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.112430 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.155826 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.197957 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.240146 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.284777 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.328781 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.374977 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.423367 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.472554 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.520623 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.573124 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.625005 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.677999 8285 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45200: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:49.704332 8523 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:49.704540 8523 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:49.705207 8523 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:49.705279 8522 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:49.705456 8522 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:49.705981 8203 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:49.706248 8203 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 8.
I20251212 08:53:49.706871 8268 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:49.713135 8523 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20251212 08:53:49.713829 8523 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:49.714340 8523 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 8 FOLLOWER]: Advancing to term 9
I20251212 08:53:49.718242 8523 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:49.719255 8523 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 9 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:49.719908 8522 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:49.720928 8203 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:49.721166 8487 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:49.721253 8203 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 8 FOLLOWER]: Advancing to term 9
I20251212 08:53:49.721414 8487 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 8.
I20251212 08:53:49.721969 8165 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:49.724679 8203 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 9.
I20251212 08:53:49.724937 8522 raft_consensus.cc:2764] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20251212 08:53:49.725873 8268 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:49.726290 8523 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 9 FOLLOWER]: Leader election won for term 9
I20251212 08:53:49.726615 8304 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:49.726794 8523 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 9 LEADER]: Becoming Leader. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Running, Role: LEADER
I20251212 08:53:49.727121 8523 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 8.14, Last appended by leader: 14, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:49.727360 8487 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:49.727485 8486 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 9 candidate_status { last_received { term: 8 index: 14 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:49.727536 8487 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 8.
I20251212 08:53:49.729741 6482 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 reported cstate change: term changed from 8 to 9, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3). New cstate: current_term: 9 leader_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: HEALTHY } } }
W20251212 08:53:49.733423 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:49.787555 8182 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43560: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:49.843709 8486 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 8 FOLLOWER]: Advancing to term 9
I20251212 08:53:49.843760 8203 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 8 index: 14. Preceding OpId from leader: term: 9 index: 16. (index mismatch)
I20251212 08:53:49.844375 8530 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20251212 08:53:49.846560 8486 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 8 index: 14. Preceding OpId from leader: term: 9 index: 16. (index mismatch)
I20251212 08:53:49.847051 8530 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20251212 08:53:49.849470 832 tablet_server.cc:178] TabletServer@127.0.208.2:42203 shutting down...
I20251212 08:53:49.862074 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:49.862453 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:49.862654 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:49.862949 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:49.863935 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:49.864140 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 LEADER]: Raft consensus shutting down.
I20251212 08:53:49.864828 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:49.881335 832 tablet_server.cc:195] TabletServer@127.0.208.2:42203 shutdown complete.
I20251212 08:53:49.885294 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:49.887739 8546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:49.887938 8547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:49.888759 832 server_base.cc:1047] running on GCE node
W20251212 08:53:49.888942 8549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:49.889319 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:49.889393 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:49.889434 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529629889417 us; error 0 us; skew 500 ppm
I20251212 08:53:49.889622 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:49.890869 832 webserver.cc:492] Webserver started at http://127.0.208.2:36849/ using document root <none> and password file <none>
I20251212 08:53:49.891105 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:49.891182 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:49.893007 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:49.894322 8554 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:49.894728 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:49.894842 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:49.894965 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:49.921372 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:49.921859 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:49.922485 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:49.923681 8561 ts_tablet_manager.cc:548] Loading tablet metadata (0/2 complete)
I20251212 08:53:49.928506 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:49.928620 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20251212 08:53:49.928735 832 ts_tablet_manager.cc:600] Registering tablets (0/2 complete)
I20251212 08:53:49.930869 8561 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:49.932576 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:49.932653 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.004s sys 0.000s
I20251212 08:53:49.946480 8561 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=10 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:49.946944 8561 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:49.947217 8561 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.006s
I20251212 08:53:49.947962 8561 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:49.948221 8561 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:49.948410 8618 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:42203 every 8 connection(s)
I20251212 08:53:49.948410 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:42203
I20251212 08:53:49.948591 8561 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 9.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:49.949329 8561 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.001s sys 0.004s
I20251212 08:53:49.949577 8561 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:49.965291 8561 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=16 overwritten=0 applied=16 ignored=0} inserts{seen=8 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:49.965759 832 tablet_server.cc:178] TabletServer@127.0.208.3:39569 shutting down...
I20251212 08:53:49.965993 8561 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:49.966281 8561 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.002s
I20251212 08:53:49.967288 8561 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:49.967582 8561 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:49.968395 8561 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 16, Last appended: 9.16, Last appended by leader: 16, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:49.969188 8561 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:53:49.974825 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:49.975283 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:49.975513 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 9 LEADER]: Raft consensus shutting down.
I20251212 08:53:49.975741 8621 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:49.975818 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 9 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:49.975871 8621 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:49.976358 8621 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:49.979672 6482 ts_manager.cc:194] Re-registered known tserver with Master: 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:49.983656 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33400
I20251212 08:53:49.984745 832 tablet_server.cc:195] TabletServer@127.0.208.3:39569 shutdown complete.
I20251212 08:53:49.985059 8621 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:49.988615 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:49.991288 8628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:49.991297 8629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:49.991907 8631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:49.991957 832 server_base.cc:1047] running on GCE node
I20251212 08:53:49.992360 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:49.992419 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:49.992444 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529629992440 us; error 0 us; skew 500 ppm
I20251212 08:53:49.992551 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:49.994303 832 webserver.cc:492] Webserver started at http://127.0.208.3:35191/ using document root <none> and password file <none>
I20251212 08:53:49.994668 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:49.994746 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:49.997895 832 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:53:49.999447 8636 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:50.000115 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:50.000222 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:50.000551 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:50.036854 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:50.037384 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:50.037955 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:50.041468 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:50.041553 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:50.043488 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:50.043555 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20251212 08:53:50.043689 8643 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap starting.
I20251212 08:53:50.060169 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39569
I20251212 08:53:50.060204 8700 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39569 every 8 connection(s)
I20251212 08:53:50.061836 8643 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=17 overwritten=0 applied=17 ignored=0} inserts{seen=9 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:50.062320 8643 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap complete.
I20251212 08:53:50.062613 8643 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent bootstrapping tablet: real 0.019s user 0.013s sys 0.003s
I20251212 08:53:50.063549 8643 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 9 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:50.063799 8643 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 9 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Initialized, Role: FOLLOWER
I20251212 08:53:50.064090 8643 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 17, Last appended: 9.17, Last appended by leader: 17, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:50.065004 8643 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251212 08:53:50.070708 8701 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:50.070866 8701 heartbeater.cc:461] Registering TS with master...
W20251212 08:53:50.070952 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:50.071259 8701 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:50.072444 6482 ts_manager.cc:194] Re-registered known tserver with Master: f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:50.073496 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33406
I20251212 08:53:50.074466 8701 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:50.080439 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.088390 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.098906 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.108965 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.115986 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.125800 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.136924 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.150135 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.166602 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.178683 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.193866 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.211295 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.228325 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.246313 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.268818 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.286940 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.305889 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.327611 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.352614 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.375679 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.403285 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.431416 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.459425 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.487751 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.516921 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.546167 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.578730 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.608713 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.642907 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.680871 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.714917 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.752069 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.791911 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.829967 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.866974 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.908015 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.949050 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:50.991048 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.037629 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.080564 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.127982 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.174734 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:51.200642 8707 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:51.200883 8707 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:51.201920 8707 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:51.216325 8712 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:51.216723 8712 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:51.220144 8594 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 10 candidate_status { last_received { term: 9 index: 19 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:51.220443 8594 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 9.
I20251212 08:53:51.220901 8409 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 10 candidate_status { last_received { term: 9 index: 19 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:51.221060 8450 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:51.221122 8409 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 9.
I20251212 08:53:51.221707 8707 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20251212 08:53:51.221916 8707 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
W20251212 08:53:51.222469 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:51.222922 8707 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Advancing to term 10
I20251212 08:53:51.225769 8707 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:51.226321 8712 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:51.227249 8594 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 10 candidate_status { last_received { term: 9 index: 19 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:51.227473 8594 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Advancing to term 10
I20251212 08:53:51.231698 8594 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 10.
I20251212 08:53:51.232148 8450 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:51.232401 8409 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 10 candidate_status { last_received { term: 9 index: 19 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:51.232635 8409 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 9 FOLLOWER]: Advancing to term 10
I20251212 08:53:51.238108 8707 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 10 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:51.242156 8409 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 10.
I20251212 08:53:51.243733 8707 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Leader election won for term 10
I20251212 08:53:51.243927 8707 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 10 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:51.245117 8707 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 9.19, Last appended by leader: 19, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:51.246477 8486 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 19 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:51.249830 6482 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 9 to 10, leader changed from 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 10 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:51.251147 8594 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 10 candidate_status { last_received { term: 9 index: 19 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:51.251448 8594 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in current term 10: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:53:51.251866 8372 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:51.252252 8712 raft_consensus.cc:2749] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20251212 08:53:51.269824 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.319278 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.369338 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.420557 8467 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42384: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:51.467399 8716 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:51.467617 8716 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:51.468413 8716 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:51.468973 8594 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 10 candidate_status { last_received { term: 9 index: 17 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
W20251212 08:53:51.474283 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:51.474303 8676 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 10 candidate_status { last_received { term: 9 index: 17 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:51.474550 8676 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 9.
I20251212 08:53:51.474974 8451 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9
I20251212 08:53:51.475235 8716 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20251212 08:53:51.475353 8716 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:51.475452 8716 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 9 FOLLOWER]: Advancing to term 10
I20251212 08:53:51.476257 8723 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:51.476428 8723 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:51.477319 8723 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:51.478461 8716 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:51.478999 8716 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 10 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:51.479496 8676 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 10 candidate_status { last_received { term: 9 index: 17 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:51.479708 8676 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 9 FOLLOWER]: Advancing to term 10
I20251212 08:53:51.480691 8594 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 10 candidate_status { last_received { term: 9 index: 17 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:51.480902 8594 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 9 FOLLOWER]: Advancing to term 10
I20251212 08:53:51.483163 8676 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 10.
I20251212 08:53:51.483620 8451 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:51.483965 8716 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Leader election won for term 10
I20251212 08:53:51.484107 8716 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 10 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:51.484557 8594 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 10.
I20251212 08:53:51.484827 8716 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 9.17, Last appended by leader: 17, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:51.487607 8486 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 17 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:51.488173 8676 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 10 candidate_status { last_received { term: 9 index: 17 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:51.488408 8676 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 10: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:53:51.488214 6482 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 9 to 10, leader changed from f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 10 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:51.488790 8558 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9; no voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86
I20251212 08:53:51.489050 8723 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20251212 08:53:51.529762 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:51.585228 8676 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 10 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 9 index: 17. Preceding OpId from leader: term: 10 index: 19. (index mismatch)
I20251212 08:53:51.585229 8594 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 9 index: 17. Preceding OpId from leader: term: 10 index: 19. (index mismatch)
I20251212 08:53:51.585834 8716 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20251212 08:53:51.586063 8707 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20251212 08:53:51.596647 832 tablet_server.cc:178] TabletServer@127.0.208.4:45411 shutting down...
I20251212 08:53:51.614143 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:51.614471 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:51.614704 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 10 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:51.614881 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 10 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:51.630072 832 tablet_server.cc:195] TabletServer@127.0.208.4:45411 shutdown complete.
I20251212 08:53:51.633698 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:51.636269 8737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:51.636541 8738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:51.636881 8740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:51.637218 832 server_base.cc:1047] running on GCE node
I20251212 08:53:51.637477 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:51.637549 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:51.637591 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529631637578 us; error 0 us; skew 500 ppm
I20251212 08:53:51.637732 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:51.639140 832 webserver.cc:492] Webserver started at http://127.0.208.4:44337/ using document root <none> and password file <none>
I20251212 08:53:51.639361 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:51.639422 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:51.641345 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:51.642620 8745 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:51.642978 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:51.643115 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:53:51.643239 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:51.659621 8594 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 9 index: 19. Preceding OpId from leader: term: 10 index: 20. (index mismatch)
I20251212 08:53:51.660208 8734 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
W20251212 08:53:51.664276 8449 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.208.4:45411: connect: Connection refused (error 111)
W20251212 08:53:51.670378 8449 consensus_peers.cc:597] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 -> Peer 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411): Couldn't send request to peer 808a87a7594d4fa0b5f8d1c78aebb8ed. Status: Network error: Client connection negotiation failed: client connection to 127.0.208.4:45411: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:51.672729 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:51.673231 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:51.673785 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:51.674906 8754 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:51.677467 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:51.677582 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:51.677719 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:51.679661 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:51.679749 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20251212 08:53:51.679832 8754 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap starting.
I20251212 08:53:51.696537 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.4:45411
I20251212 08:53:51.696557 8811 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.4:45411 every 8 connection(s)
I20251212 08:53:51.703644 8754 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=19 overwritten=0 applied=19 ignored=0} inserts{seen=10 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:51.704237 8754 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap complete.
I20251212 08:53:51.704573 8754 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent bootstrapping tablet: real 0.025s user 0.024s sys 0.000s
I20251212 08:53:51.705433 8754 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 10 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:51.705670 8754 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Initialized, Role: FOLLOWER
I20251212 08:53:51.705978 8754 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 19, Last appended: 9.19, Last appended by leader: 19, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:51.706759 832 tablet_server.cc:178] TabletServer@127.0.208.1:35903 shutting down...
I20251212 08:53:51.713586 8754 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent starting tablet: real 0.009s user 0.002s sys 0.007s
I20251212 08:53:51.722563 8812 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:51.722779 8812 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:51.723158 8812 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:51.724067 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:51.724416 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:51.724428 6480 ts_manager.cc:194] Re-registered known tserver with Master: 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411)
I20251212 08:53:51.724651 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 10 LEADER]: Raft consensus shutting down.
I20251212 08:53:51.724944 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:51.725749 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:51.725752 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33416
I20251212 08:53:51.725881 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 10 LEADER]: Raft consensus shutting down.
I20251212 08:53:51.726167 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:51.726974 8812 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:51.753540 832 tablet_server.cc:195] TabletServer@127.0.208.1:35903 shutdown complete.
I20251212 08:53:51.757763 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:51.760375 8818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:51.760445 8817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:51.761544 832 server_base.cc:1047] running on GCE node
W20251212 08:53:51.761622 8820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:51.761947 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:51.762006 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:51.762032 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529631762028 us; error 0 us; skew 500 ppm
I20251212 08:53:51.762158 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:51.763437 832 webserver.cc:492] Webserver started at http://127.0.208.1:36883/ using document root <none> and password file <none>
I20251212 08:53:51.763618 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:51.763664 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:51.765574 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:51.767102 8825 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:51.767571 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:51.767678 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:51.767850 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:51.785805 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:51.786300 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:51.786868 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:51.793005 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:51.793103 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20251212 08:53:51.795135 8832 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:51.796826 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:51.796890 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:51.810595 8832 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=20 overwritten=0 applied=20 ignored=0} inserts{seen=10 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:51.811012 8832 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:51.811260 8832 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.016s user 0.013s sys 0.000s
I20251212 08:53:51.811911 8832 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:51.812067 8832 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:51.812230 8889 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35903 every 8 connection(s)
I20251212 08:53:51.812225 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35903
I20251212 08:53:51.812331 8832 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 10.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:51.812927 8832 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251212 08:53:51.813203 8832 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:51.820483 8891 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:51.820636 8891 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:51.820982 8891 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:51.822309 6480 ts_manager.cc:194] Re-registered known tserver with Master: 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:51.824007 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33426
W20251212 08:53:51.829035 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:51.830751 8832 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=21 overwritten=1 applied=20 ignored=0} inserts{seen=11 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:51.831142 8832 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:51.831365 8832 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.018s user 0.014s sys 0.000s
I20251212 08:53:51.831969 8832 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:51.832118 8832 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:51.832369 8832 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 10.20, Last appended by leader: 20, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:51.832747 8891 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:51.832785 8832 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
W20251212 08:53:51.834888 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.844985 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.853171 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.862509 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.872958 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.883904 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.898025 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.910547 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.923743 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.939857 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.959404 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.974529 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:51.990675 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.011099 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.030169 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.049329 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.075242 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.100533 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.123705 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.149276 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.175443 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.201689 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.229230 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.256237 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.287422 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.320206 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.351418 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.386660 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.422183 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.456331 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.493554 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.532258 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.568315 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.606630 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.650220 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.692498 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.735357 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.779094 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.825114 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.868351 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.918015 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:52.966212 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.012542 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.062104 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.112247 8574 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:43586: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:53.165656 8897 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
W20251212 08:53:53.165664 8656 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:45234: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:53.165855 8897 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:53.166668 8897 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:53.184564 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:53.184576 8865 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:53.184831 8787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 10.
I20251212 08:53:53.184839 8865 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 10.
I20251212 08:53:53.185245 8556 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters:
I20251212 08:53:53.185487 8897 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20251212 08:53:53.185575 8897 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:53.185660 8897 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Advancing to term 11
I20251212 08:53:53.187824 8897 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:53.188407 8897 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 11 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:53.188784 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:53.188810 8865 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:53.188948 8787 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 10 FOLLOWER]: Advancing to term 11
I20251212 08:53:53.188958 8865 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Advancing to term 11
I20251212 08:53:53.190989 8865 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 11.
I20251212 08:53:53.190985 8787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 11.
I20251212 08:53:53.191377 8558 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:53.191632 8897 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Leader election won for term 11
I20251212 08:53:53.192057 8897 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:53:53.192360 8897 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 10.20, Last appended by leader: 20, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:53.194839 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 10 to 11, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2). New cstate: current_term: 11 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:53.204968 8908 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:53.205152 8908 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:53.205928 8908 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:53.206236 8897 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:53.206404 8897 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:53.207137 8897 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:53.207747 8865 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:53.207943 8865 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 10.
I20251212 08:53:53.208302 8676 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:53.208462 8676 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 10.
I20251212 08:53:53.208554 8558 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:53.208882 8897 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20251212 08:53:53.208984 8897 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:53.209056 8897 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 10 FOLLOWER]: Advancing to term 11
I20251212 08:53:53.211761 8897 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:53.212369 8897 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 11 election: Requested vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:53.212733 8865 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:53.212816 8676 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:53.212881 8865 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 10 FOLLOWER]: Advancing to term 11
I20251212 08:53:53.212919 8676 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 10 FOLLOWER]: Advancing to term 11
I20251212 08:53:53.214358 8864 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:53.214386 8594 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 11 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:53.214666 8594 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f2ceec6428fb4ee0b98c750a718d3a86 in current term 11: Already voted for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in this term.
I20251212 08:53:53.215024 8639 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2ceec6428fb4ee0b98c750a718d3a86; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:53.215772 8676 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 11.
I20251212 08:53:53.215997 8908 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20251212 08:53:53.216010 8865 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 11.
I20251212 08:53:53.216202 8558 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:53.216451 8897 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Leader election won for term 11
I20251212 08:53:53.216574 8897 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:53:53.216866 8897 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 10.20, Last appended by leader: 20, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
W20251212 08:53:53.219272 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:53.219302 6480 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 10 to 11, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2). New cstate: current_term: 11 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:53.275328 8676 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 11 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 20. Preceding OpId from leader: term: 11 index: 22. (index mismatch)
I20251212 08:53:53.275345 8865 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 10 index: 20. Preceding OpId from leader: term: 11 index: 22. (index mismatch)
I20251212 08:53:53.275915 8902 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20251212 08:53:53.277480 8897 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20251212 08:53:53.283902 832 tablet_server.cc:178] TabletServer@127.0.208.2:42203 shutting down...
I20251212 08:53:53.291567 8915 mvcc.cc:204] Tried to move back new op lower bound from 7231609377890754560 to 7231609377658531840. Current Snapshot: MvccSnapshot[applied={T|T < 7231609377890754560}]
I20251212 08:53:53.297362 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:53.298130 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:53.298298 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 LEADER]: Raft consensus shutting down.
I20251212 08:53:53.298579 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:53.299340 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:53.299463 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 LEADER]: Raft consensus shutting down.
I20251212 08:53:53.299660 832 pending_rounds.cc:70] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Trying to abort 1 pending ops.
I20251212 08:53:53.299702 832 pending_rounds.cc:77] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Aborting op as it isn't in flight: id { term: 11 index: 21 } timestamp: 7231609377558323200 op_type: NO_OP noop_request { }
I20251212 08:53:53.299801 832 raft_consensus.cc:2889] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251212 08:53:53.299901 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:53.316241 832 tablet_server.cc:195] TabletServer@127.0.208.2:42203 shutdown complete.
I20251212 08:53:53.320042 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:53.322597 8925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:53.322682 8924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:53.323007 8927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:53.323354 832 server_base.cc:1047] running on GCE node
I20251212 08:53:53.323637 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:53.323697 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:53.323720 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529633323718 us; error 0 us; skew 500 ppm
I20251212 08:53:53.323868 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:53.325140 832 webserver.cc:492] Webserver started at http://127.0.208.2:36849/ using document root <none> and password file <none>
I20251212 08:53:53.325344 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:53.325395 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:53.327158 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:53.328533 8932 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:53.328879 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:53.328985 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:53.329097 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:53.339447 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:53.340101 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:53.341006 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:53.342540 8939 ts_tablet_manager.cc:548] Loading tablet metadata (0/2 complete)
I20251212 08:53:53.348505 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:53.348601 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20251212 08:53:53.348670 832 ts_tablet_manager.cc:600] Registering tablets (0/2 complete)
I20251212 08:53:53.351349 8939 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:53.353073 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:53.353135 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.004s sys 0.000s
I20251212 08:53:53.367774 8939 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=21 overwritten=0 applied=20 ignored=0} inserts{seen=10 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251212 08:53:53.368237 8939 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:53.368528 8939 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.017s user 0.017s sys 0.000s
I20251212 08:53:53.369329 8939 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:53.369606 8939 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:53.369848 8939 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 20, Last appended: 11.21, Last appended by leader: 21, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:53.370395 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:42203
I20251212 08:53:53.370458 8997 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:42203 every 8 connection(s)
I20251212 08:53:53.370621 8939 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:53.370847 8939 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:53.383998 8998 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:53.384156 8998 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:53.384495 8998 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:53.385710 6482 ts_manager.cc:194] Re-registered known tserver with Master: 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:53.387362 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46096
W20251212 08:53:53.388164 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:53.393909 8939 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=22 overwritten=0 applied=22 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:53.394312 8939 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:53.394538 8939 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.004s
I20251212 08:53:53.395169 8939 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:53.395318 8939 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:53.395565 8939 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 22, Last appended: 11.22, Last appended by leader: 22, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:53.395964 8998 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:53.396098 8939 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20251212 08:53:53.398729 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.406456 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.413596 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.420512 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.430330 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.442652 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.451417 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.461319 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.473896 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.486987 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.501982 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.518436 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.537719 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.556918 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.575534 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.594681 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.615833 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.637748 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.660995 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.684216 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.712800 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.737680 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.765903 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.796847 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.825116 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.853298 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.885851 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.916801 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:53.951839 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:53.989548 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.026715 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:54.061864 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.098495 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.136746 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:54.177983 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.220464 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.261590 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:54.305429 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.347710 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.389863 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:54.434854 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.482689 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:54.521821 9005 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:54.522007 9005 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:54.522814 9005 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
W20251212 08:53:54.528602 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:54.529197 8676 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 22 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:54.529978 8865 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 12 candidate_status { last_received { term: 11 index: 22 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:54.530375 8936 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9; no voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86
I20251212 08:53:54.530630 9005 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20251212 08:53:54.578861 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.630225 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.678329 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:54.727391 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.782122 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:54.784540 9010 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:54.784719 9010 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:54.785439 9010 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:54.791049 8676 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 12 candidate_status { last_received { term: 11 index: 22 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:54.791049 8972 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 12 candidate_status { last_received { term: 11 index: 22 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:54.791309 8676 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 11.
I20251212 08:53:54.791375 8972 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 11.
I20251212 08:53:54.791700 8829 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:54.791971 9010 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20251212 08:53:54.792054 9010 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:54.792133 9010 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Advancing to term 12
I20251212 08:53:54.794241 9010 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:54.794816 9010 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 12 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:54.795256 8972 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 12 candidate_status { last_received { term: 11 index: 22 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:54.795271 8676 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 12 candidate_status { last_received { term: 11 index: 22 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:54.795397 8972 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Advancing to term 12
I20251212 08:53:54.795467 8676 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 11 FOLLOWER]: Advancing to term 12
I20251212 08:53:54.797534 8676 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 12.
I20251212 08:53:54.797535 8972 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 12.
I20251212 08:53:54.797941 8829 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:54.798204 9010 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Leader election won for term 12
I20251212 08:53:54.798617 9010 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 12 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:54.798872 9010 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 11.22, Last appended by leader: 22, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:54.801118 6480 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 11 to 12, leader changed from 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 12 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: UNKNOWN } } }
W20251212 08:53:54.835268 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:54.887887 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.943691 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:54.997880 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:55.056967 8845 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:55.066004 9020 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:55.066210 9020 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.066957 9020 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:55.074101 8865 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 19 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:55.074330 8865 raft_consensus.cc:2410] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 808a87a7594d4fa0b5f8d1c78aebb8ed for term 12 because replica has last-logged OpId of term: 10 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 19.
I20251212 08:53:55.074579 8972 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 12 candidate_status { last_received { term: 9 index: 19 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:55.074774 8972 raft_consensus.cc:2410] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 808a87a7594d4fa0b5f8d1c78aebb8ed for term 12 because replica has last-logged OpId of term: 11 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 19.
I20251212 08:53:55.075079 8748 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:55.075331 9020 raft_consensus.cc:2749] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20251212 08:53:55.115763 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:55.167044 9010 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:55.167263 9010 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.168011 9010 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:55.168613 8972 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 12 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:55.168870 8972 raft_consensus.cc:2410] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 for term 12 because replica has last-logged OpId of term: 11 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 20.
W20251212 08:53:55.172735 8767 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34784: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
I20251212 08:53:55.173347 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 12 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:55.173506 8787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 11.
I20251212 08:53:55.173883 8827 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9
I20251212 08:53:55.174134 9010 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20251212 08:53:55.174217 9010 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:55.174297 9010 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 11 FOLLOWER]: Advancing to term 12
I20251212 08:53:55.176470 9010 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.176986 9010 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 12 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:55.177407 8972 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 12 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:55.177408 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 12 candidate_status { last_received { term: 10 index: 20 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:55.177616 8972 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 11 FOLLOWER]: Advancing to term 12
I20251212 08:53:55.177623 8787 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 11 FOLLOWER]: Advancing to term 12
I20251212 08:53:55.180609 8787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 12.
I20251212 08:53:55.180706 8972 raft_consensus.cc:2410] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 for term 12 because replica has last-logged OpId of term: 11 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 20.
I20251212 08:53:55.181109 8827 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9
I20251212 08:53:55.181397 9010 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Leader election won for term 12
I20251212 08:53:55.181548 9010 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 12 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:55.182000 9010 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 10.20, Last appended by leader: 20, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.184501 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 11 to 12, leader changed from 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 12 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:55.204491 9026 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:53:55.204677 9026 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.205397 9026 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:55.205936 8865 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 21 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:55.210588 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 21 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:55.210769 8787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 12.
I20251212 08:53:55.211091 8934 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:55.211350 9026 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20251212 08:53:55.211436 9026 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:53:55.211501 9026 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Advancing to term 13
I20251212 08:53:55.213641 9026 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.214193 9026 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 13 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:55.214591 8787 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 21 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:55.214608 8865 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 13 candidate_status { last_received { term: 11 index: 21 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:55.214735 8787 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 12 FOLLOWER]: Advancing to term 13
I20251212 08:53:55.216925 8787 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 13.
I20251212 08:53:55.217267 8934 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:55.217514 9026 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Leader election won for term 13
I20251212 08:53:55.217871 9026 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:53:55.218106 9026 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 11.21, Last appended by leader: 21, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.220340 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 12 to 13, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2). New cstate: current_term: 13 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:55.222361 8676 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 11 index: 22. Preceding OpId from leader: term: 12 index: 23. (index mismatch)
I20251212 08:53:55.222895 9010 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20251212 08:53:55.227241 8972 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 11 index: 22. Preceding OpId from leader: term: 12 index: 23. (index mismatch)
I20251212 08:53:55.227811 9034 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20251212 08:53:55.234086 8787 raft_consensus.cc:1240] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Rejecting Update request from peer 8a42a66698e144409d7216b8f7b78219 for earlier term 12. Current term is 13. Ops: []
I20251212 08:53:55.234587 8972 raft_consensus.cc:1240] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 LEADER]: Rejecting Update request from peer 8a42a66698e144409d7216b8f7b78219 for earlier term 12. Current term is 13. Ops: []
I20251212 08:53:55.234696 9034 consensus_queue.cc:1059] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.000s
I20251212 08:53:55.234961 9010 raft_consensus.cc:3055] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 12 LEADER]: Stepping down as leader of term 12
I20251212 08:53:55.235045 9010 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:55.235234 9034 consensus_queue.cc:1059] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20251212 08:53:55.235383 9034 consensus_queue.cc:1243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Peer 1b32dae091b44ce9a6fe4ae655cf01b9 log is divergent from this leader: its last log entry 13.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20251212 08:53:55.235599 9010 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 12.22, Last appended by leader: 22, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.235877 9010 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Advancing to term 13
I20251212 08:53:55.602430 8865 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 12 index: 22. Preceding OpId from leader: term: 13 index: 22. (term mismatch)
I20251212 08:53:55.602579 8865 pending_rounds.cc:85] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Aborting all ops after (but not including) 21
I20251212 08:53:55.602620 8865 pending_rounds.cc:107] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Aborting uncommitted WRITE_OP operation due to leader change: 12.22
W20251212 08:53:55.602841 8865 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:42414: Aborted: Op aborted by new leader
I20251212 08:53:55.603825 9026 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20251212 08:53:55.604115 9026 consensus_queue.cc:1243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Peer 8a42a66698e144409d7216b8f7b78219 log is divergent from this leader: its last log entry 12.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20251212 08:53:55.606288 8865 pending_rounds.cc:85] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Aborting all ops after (but not including) 20
I20251212 08:53:55.606352 8865 pending_rounds.cc:107] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Aborting uncommitted NO_OP operation due to leader change: 12.21
I20251212 08:53:55.606388 8865 raft_consensus.cc:2889] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20251212 08:53:55.610227 8787 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 9 index: 19. Preceding OpId from leader: term: 13 index: 22. (index mismatch)
I20251212 08:53:55.610741 9037 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 19, Time since last communication: 0.000s
I20251212 08:53:55.668176 832 tablet_server.cc:178] TabletServer@127.0.208.3:39569 shutting down...
I20251212 08:53:55.678683 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:55.678987 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:55.679169 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:55.679420 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:55.693702 832 tablet_server.cc:195] TabletServer@127.0.208.3:39569 shutdown complete.
I20251212 08:53:55.697175 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:55.699450 9045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:55.699671 9046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:55.699975 9048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:55.700402 832 server_base.cc:1047] running on GCE node
I20251212 08:53:55.700667 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:55.700721 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:55.700744 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529635700741 us; error 0 us; skew 500 ppm
I20251212 08:53:55.700846 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:55.701968 832 webserver.cc:492] Webserver started at http://127.0.208.3:35191/ using document root <none> and password file <none>
I20251212 08:53:55.702133 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:55.702176 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:55.703861 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:55.705197 9053 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:55.705598 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:55.705684 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:55.705778 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:55.717476 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:55.717957 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:55.718406 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:55.719512 9060 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:55.722383 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:55.722455 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:55.722524 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:55.724622 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:55.724679 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:55.724793 9060 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap starting.
I20251212 08:53:55.738632 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39569
I20251212 08:53:55.738648 9117 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39569 every 8 connection(s)
W20251212 08:53:55.741322 8952 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46120: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:55.747047 9060 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:55.747514 9060 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap complete.
I20251212 08:53:55.747772 9060 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent bootstrapping tablet: real 0.023s user 0.019s sys 0.000s
I20251212 08:53:55.748385 9060 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:55.748526 9060 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Initialized, Role: FOLLOWER
I20251212 08:53:55.748768 9060 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 12.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:55.750169 9060 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent starting tablet: real 0.002s user 0.006s sys 0.000s
I20251212 08:53:55.751441 9118 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:55.751570 9118 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:55.751995 9118 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:55.753062 6482 ts_manager.cc:194] Re-registered known tserver with Master: f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:55.754323 832 tablet_server.cc:178] TabletServer@127.0.208.4:45411 shutting down...
I20251212 08:53:55.754482 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46102
I20251212 08:53:55.755715 9118 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:55.765065 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:55.765393 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:55.765630 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:55.765836 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:55.771396 832 tablet_server.cc:195] TabletServer@127.0.208.4:45411 shutdown complete.
I20251212 08:53:55.774727 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:55.777067 9128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:55.777506 9129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:55.777509 9131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:55.777621 832 server_base.cc:1047] running on GCE node
I20251212 08:53:55.778090 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:55.778156 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:55.778191 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529635778188 us; error 0 us; skew 500 ppm
I20251212 08:53:55.778291 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:55.779381 832 webserver.cc:492] Webserver started at http://127.0.208.4:44337/ using document root <none> and password file <none>
I20251212 08:53:55.779542 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:55.779585 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:55.781282 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20251212 08:53:55.782402 9136 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:55.782719 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:53:55.782816 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:53:55.782917 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:55.810109 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:55.810639 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:55.811177 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:55.814728 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:55.814821 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:55.816881 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:55.816943 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:55.817061 9143 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap starting.
I20251212 08:53:55.833345 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.4:45411
I20251212 08:53:55.833345 9200 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.4:45411 every 8 connection(s)
I20251212 08:53:55.836890 9143 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=23 overwritten=0 applied=23 ignored=0} inserts{seen=11 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:55.838761 9143 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap complete.
I20251212 08:53:55.839063 9143 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.005s
I20251212 08:53:55.839879 9143 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.840070 9143 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Initialized, Role: FOLLOWER
I20251212 08:53:55.840342 9143 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 13.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.840591 832 tablet_server.cc:178] TabletServer@127.0.208.1:35903 shutting down...
I20251212 08:53:55.845677 9143 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20251212 08:53:55.845892 9201 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:55.846098 9201 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:55.846554 9201 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:55.847911 6482 ts_manager.cc:194] Re-registered known tserver with Master: 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411)
I20251212 08:53:55.850379 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46114
I20251212 08:53:55.852062 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:55.852099 9201 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:55.852346 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:55.852533 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 12 LEADER]: Raft consensus shutting down.
I20251212 08:53:55.852880 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:55.853642 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:55.853767 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:55.853938 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:55.858659 832 tablet_server.cc:195] TabletServer@127.0.208.1:35903 shutdown complete.
I20251212 08:53:55.862138 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:55.864557 9206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:55.864583 9205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:55.865187 9208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:55.865422 832 server_base.cc:1047] running on GCE node
I20251212 08:53:55.873667 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:55.873766 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:55.873808 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529635873803 us; error 0 us; skew 500 ppm
I20251212 08:53:55.873961 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:55.876158 832 webserver.cc:492] Webserver started at http://127.0.208.1:36883/ using document root <none> and password file <none>
I20251212 08:53:55.876444 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:55.876514 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:55.878764 832 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:55.880141 9213 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:55.880535 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:55.880638 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:55.880749 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:55.895457 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:55.895936 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:55.896523 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:55.902554 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:55.902653 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20251212 08:53:55.904819 9220 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:55.906368 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:55.906443 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.002s sys 0.000s
I20251212 08:53:55.921766 9220 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=25 overwritten=2 applied=23 ignored=0} inserts{seen=11 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:55.921862 9277 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35903 every 8 connection(s)
I20251212 08:53:55.921857 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35903
I20251212 08:53:55.922206 9220 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:55.922466 9220 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.018s user 0.015s sys 0.000s
I20251212 08:53:55.924125 9220 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.924355 9220 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:55.924686 9220 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 23, Last appended: 13.23, Last appended by leader: 23, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:55.925418 9220 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20251212 08:53:55.925717 9220 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:55.930681 9278 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:55.930835 9278 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:55.931156 9278 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:55.932534 6482 ts_manager.cc:194] Re-registered known tserver with Master: 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:55.934877 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46122
I20251212 08:53:55.936390 9278 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:55.949874 832 tablet_server.cc:178] TabletServer@127.0.208.2:42203 shutting down...
I20251212 08:53:55.958264 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:55.958586 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:55.958820 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:55.959031 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:55.959874 9220 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=26 overwritten=1 applied=25 ignored=0} inserts{seen=14 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:55.959889 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:55.960068 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 LEADER]: Raft consensus shutting down.
I20251212 08:53:55.960348 9220 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:55.960383 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:55.960603 9220 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.035s user 0.026s sys 0.005s
I20251212 08:53:55.961342 9220 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:55.961545 9220 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:55.961805 9220 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 12.25, Last appended by leader: 25, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:55.962591 9220 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20251212 08:53:55.964020 9216 connection.cc:537] server connection from 127.0.0.1:58654 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251212 08:53:55.966432 832 tablet_server.cc:195] TabletServer@127.0.208.2:42203 shutdown complete.
I20251212 08:53:55.970608 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:55.973229 9294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:55.973258 9293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:55.973704 832 server_base.cc:1047] running on GCE node
W20251212 08:53:55.973738 9296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:55.974082 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:55.974140 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:55.974164 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529635974161 us; error 0 us; skew 500 ppm
I20251212 08:53:55.974284 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:55.975488 832 webserver.cc:492] Webserver started at http://127.0.208.2:36849/ using document root <none> and password file <none>
I20251212 08:53:55.975667 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:55.975710 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:55.977514 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:55.978762 9301 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:55.979070 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:55.979166 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:55.979271 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:55.992981 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:55.993481 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:55.993991 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:55.999636 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:55.999756 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20251212 08:53:56.002135 9308 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:56.003932 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:56.003998 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20251212 08:53:56.018065 9365 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:42203 every 8 connection(s)
I20251212 08:53:56.018061 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:42203
W20251212 08:53:56.026120 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:56.026288 9366 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:56.026422 9366 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:56.026770 9366 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:56.027770 9308 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=24 overwritten=0 applied=24 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:56.027942 6482 ts_manager.cc:194] Re-registered known tserver with Master: 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:56.028153 9308 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:56.028407 9308 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.026s user 0.021s sys 0.004s
I20251212 08:53:56.028993 9308 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:56.029182 9308 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:56.029488 6482 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46134
I20251212 08:53:56.029413 9308 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 24, Last appended: 13.24, Last appended by leader: 24, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:56.030227 9308 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:56.030514 9308 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:56.031116 9366 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:56.045784 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:56.051466 9308 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=25 overwritten=0 applied=25 ignored=0} inserts{seen=14 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20251212 08:53:56.051774 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:56.051870 9308 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:56.052106 9308 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.004s
I20251212 08:53:56.052819 9308 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:56.052961 9308 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:56.053157 9308 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 12.25, Last appended by leader: 25, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:56.053536 9308 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.001s user 0.004s sys 0.000s
W20251212 08:53:56.061478 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.068197 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.078576 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.089253 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.098991 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.115290 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.128134 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.143249 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.161077 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.180390 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.197495 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.216979 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.238930 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.258790 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.283299 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.307205 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.331198 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.359845 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.387868 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.415771 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.444171 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.474278 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.505332 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.537734 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.571585 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.605644 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.642156 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.678180 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.716225 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.752878 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.789038 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.829250 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.869843 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.911870 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.953557 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:56.999047 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.044019 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.087167 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.132910 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.177927 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.228112 9073 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38286: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.279879 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.329061 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:57.340904 9373 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:57.341138 9373 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:57.342363 9373 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:57.349496 9341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:57.349776 9341 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 12.
I20251212 08:53:57.350339 9056 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:57.350631 9373 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20251212 08:53:57.350719 9373 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 8a42a66698e144409d7216b8f7b78219)
I20251212 08:53:57.350812 9373 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 12 FOLLOWER]: Advancing to term 13
I20251212 08:53:57.351320 9253 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:57.351480 9253 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 12.
I20251212 08:53:57.353302 9373 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:57.353813 9373 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 13 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:57.354180 9341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:57.354336 9341 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 12 FOLLOWER]: Advancing to term 13
I20251212 08:53:57.355806 9253 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" candidate_term: 13 candidate_status { last_received { term: 12 index: 25 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:57.355970 9253 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 12 FOLLOWER]: Advancing to term 13
I20251212 08:53:57.357007 9341 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 13.
I20251212 08:53:57.357396 9056 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:57.357662 9373 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 13 FOLLOWER]: Leader election won for term 13
I20251212 08:53:57.358098 9373 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 13 LEADER]: Becoming Leader. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Running, Role: LEADER
I20251212 08:53:57.358392 9373 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 12.25, Last appended by leader: 25, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:57.359050 9253 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2ceec6428fb4ee0b98c750a718d3a86 in term 13.
I20251212 08:53:57.362092 6482 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 reported cstate change: term changed from 12 to 13, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3). New cstate: current_term: 13 leader_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: HEALTHY } } }
I20251212 08:53:57.383327 9341 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 12 index: 25. Preceding OpId from leader: term: 13 index: 27. (index mismatch)
I20251212 08:53:57.383340 9253 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Refusing update from remote peer f2ceec6428fb4ee0b98c750a718d3a86: Log matching property violated. Preceding OpId in replica: term: 12 index: 25. Preceding OpId from leader: term: 13 index: 27. (index mismatch)
I20251212 08:53:57.383909 9378 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251212 08:53:57.384057 9373 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251212 08:53:57.391525 832 tablet_server.cc:178] TabletServer@127.0.208.3:39569 shutting down...
I20251212 08:53:57.408869 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:57.409212 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:57.409408 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 13 LEADER]: Raft consensus shutting down.
I20251212 08:53:57.409739 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 13 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:57.426182 832 tablet_server.cc:195] TabletServer@127.0.208.3:39569 shutdown complete.
I20251212 08:53:57.429888 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:57.432408 9395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:57.432574 9396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:57.432832 9398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:57.433146 832 server_base.cc:1047] running on GCE node
I20251212 08:53:57.433400 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:57.433485 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:57.433552 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529637433533 us; error 0 us; skew 500 ppm
I20251212 08:53:57.433723 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:57.434896 832 webserver.cc:492] Webserver started at http://127.0.208.3:35191/ using document root <none> and password file <none>
I20251212 08:53:57.435103 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:57.435178 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:57.436928 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:53:57.438081 9403 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:57.438396 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:57.438516 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:57.438638 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:57.448453 9385 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:57.448679 9385 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:57.449573 9385 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:57.460925 9411 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:57.461310 9411 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:57.464555 9411 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:57.467512 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:57.468077 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:57.469182 9176 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 14 candidate_status { last_received { term: 13 index: 24 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:53:57.469518 9176 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 13.
I20251212 08:53:57.470263 9215 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:57.470667 9385 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20251212 08:53:57.470783 9385 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 1b32dae091b44ce9a6fe4ae655cf01b9)
I20251212 08:53:57.470903 9385 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Advancing to term 14
I20251212 08:53:57.475333 9385 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:57.476938 9385 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 14 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:57.477734 9176 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 14 candidate_status { last_received { term: 13 index: 24 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:53:57.477979 9176 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 13 FOLLOWER]: Advancing to term 14
I20251212 08:53:57.480975 9341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 14 candidate_status { last_received { term: 13 index: 24 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:57.481245 9341 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 13.
I20251212 08:53:57.482082 9341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 14 candidate_status { last_received { term: 13 index: 24 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:57.482306 9341 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Advancing to term 14
I20251212 08:53:57.482929 9176 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 14.
I20251212 08:53:57.483501 9215 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:57.483985 9385 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Leader election won for term 14
I20251212 08:53:57.485898 9385 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 14 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:53:57.486397 9385 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 13.24, Last appended by leader: 24, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:57.486668 9341 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 14.
I20251212 08:53:57.490176 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:57.491202 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 13 to 14, leader changed from 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 14 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:57.494207 9253 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 24 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:57.495231 9419 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:57.496181 9341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 14 candidate_status { last_received { term: 13 index: 24 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:57.496436 9341 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in current term 14: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:53:57.496888 9139 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:57.497210 9411 raft_consensus.cc:2749] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20251212 08:53:57.498137 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:57.498220 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.002s sys 0.000s
I20251212 08:53:57.498353 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:57.500308 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:57.500522 9419 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap starting.
I20251212 08:53:57.500528 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:57.517421 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39569
I20251212 08:53:57.517450 9476 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39569 every 8 connection(s)
I20251212 08:53:57.525801 9419 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=15 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:57.526557 9419 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap complete.
I20251212 08:53:57.527702 9419 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent bootstrapping tablet: real 0.027s user 0.024s sys 0.000s
I20251212 08:53:57.528740 9419 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 13 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:57.529042 9419 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 13 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Initialized, Role: FOLLOWER
I20251212 08:53:57.529428 9419 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 13.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:57.530277 9419 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:57.534005 9477 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:57.534189 9477 heartbeater.cc:461] Registering TS with master...
W20251212 08:53:57.534457 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:53:57.534524 9477 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:57.535797 6480 ts_manager.cc:194] Re-registered known tserver with Master: f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:57.537019 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46138
I20251212 08:53:57.538115 9477 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:57.540248 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.545890 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.553913 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.560583 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.568454 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.580175 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.593243 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.605355 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.617866 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.632997 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.648192 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.665757 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.683792 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.699858 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.719417 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.739414 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.761511 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.787333 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.812431 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.835610 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.861186 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.889302 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.917415 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:57.931551 9341 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 13 index: 24. Preceding OpId from leader: term: 14 index: 25. (index mismatch)
I20251212 08:53:57.932093 9385 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
I20251212 08:53:57.937405 9176 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 13 index: 24. Preceding OpId from leader: term: 14 index: 25. (index mismatch)
I20251212 08:53:57.937996 9489 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 25, Last known committed idx: 24, Time since last communication: 0.000s
W20251212 08:53:57.944461 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:57.975570 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.006492 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.038893 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.072016 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.106395 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.140049 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.173328 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.208644 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.249851 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.286073 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.325201 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.364813 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.405905 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.449275 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.496414 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.542186 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.588227 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.637945 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.687125 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.734196 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.786983 9432 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38302: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.835085 9321 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46156: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:58.884203 9233 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58670: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:58.891604 9494 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:58.891793 9494 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:58.892710 9494 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:58.894389 9493 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:58.894570 9493 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:58.895502 9493 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:58.895890 9341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 14 candidate_status { last_received { term: 13 index: 27 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:53:58.896085 9341 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 13.
I20251212 08:53:58.896597 9216 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:53:58.897092 9493 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20251212 08:53:58.897192 9493 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:58.897281 9493 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 13 FOLLOWER]: Advancing to term 14
I20251212 08:53:58.901407 9452 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 27 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:58.901603 9452 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 13.
I20251212 08:53:58.901667 9493 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:58.902024 9305 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:53:58.902256 9494 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20251212 08:53:58.902333 9494 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f2ceec6428fb4ee0b98c750a718d3a86)
I20251212 08:53:58.902417 9494 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 13 FOLLOWER]: Advancing to term 14
I20251212 08:53:58.902830 9341 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 14 candidate_status { last_received { term: 13 index: 27 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:53:58.902897 9493 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 14 election: Requested vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:58.903684 9253 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 27 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:53:58.903877 9253 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 14: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:53:58.905210 9494 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:58.905503 9341 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 in current term 14: Already voted for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in this term.
I20251212 08:53:58.905827 9494 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 14 election: Requested vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:53:58.906261 9452 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 27 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:58.906409 9452 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 13 FOLLOWER]: Advancing to term 14
I20251212 08:53:58.906409 9253 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 14 candidate_status { last_received { term: 13 index: 27 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:53:58.906617 9253 raft_consensus.cc:2393] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 14: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:53:58.909065 9451 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 14 candidate_status { last_received { term: 13 index: 27 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:53:58.909183 9450 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 14 candidate_status { last_received { term: 13 index: 27 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:53:58.909513 9452 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 14.
I20251212 08:53:58.909765 9217 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86
I20251212 08:53:58.909801 9305 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters: 8a42a66698e144409d7216b8f7b78219
I20251212 08:53:58.910169 9493 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20251212 08:53:58.910341 9494 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Leader election won for term 14
I20251212 08:53:58.910672 9494 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:53:58.910903 9494 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 13.27, Last appended by leader: 27, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:58.913316 6482 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 13 to 14, leader changed from f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3) to 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2). New cstate: current_term: 14 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: UNKNOWN } } }
I20251212 08:53:58.938174 9452 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 14 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 27. Preceding OpId from leader: term: 14 index: 29. (index mismatch)
I20251212 08:53:58.938174 9253 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 13 index: 27. Preceding OpId from leader: term: 14 index: 29. (index mismatch)
I20251212 08:53:58.938791 9501 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251212 08:53:58.938975 9494 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251212 08:53:58.954054 832 tablet_server.cc:178] TabletServer@127.0.208.4:45411 shutting down...
I20251212 08:53:58.963958 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:58.964258 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:58.964457 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:58.964654 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:58.978406 832 tablet_server.cc:195] TabletServer@127.0.208.4:45411 shutdown complete.
I20251212 08:53:58.981931 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:58.984530 9517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:58.984565 9518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:58.984910 9520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:58.985916 832 server_base.cc:1047] running on GCE node
I20251212 08:53:58.986138 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:58.986202 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:58.986228 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529638986225 us; error 0 us; skew 500 ppm
I20251212 08:53:58.986326 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:58.987763 832 webserver.cc:492] Webserver started at http://127.0.208.4:44337/ using document root <none> and password file <none>
I20251212 08:53:58.987954 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:58.988000 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:58.989821 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:58.991181 9525 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:58.991542 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:58.991650 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:53:58.991760 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:59.021785 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:59.022379 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:59.022974 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:59.024154 9532 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:53:59.026667 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:53:59.026773 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:53:59.026883 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:53:59.029034 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:53:59.029116 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20251212 08:53:59.029290 9532 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap starting.
I20251212 08:53:59.046005 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.4:45411
I20251212 08:53:59.046005 9589 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.4:45411 every 8 connection(s)
I20251212 08:53:59.048535 9532 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=25 overwritten=0 applied=25 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:59.049088 9532 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap complete.
I20251212 08:53:59.049829 9532 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.000s
I20251212 08:53:59.051204 9532 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:59.051494 9532 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Initialized, Role: FOLLOWER
I20251212 08:53:59.051795 9532 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 14.25, Last appended by leader: 25, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:59.053709 832 tablet_server.cc:178] TabletServer@127.0.208.1:35903 shutting down...
I20251212 08:53:59.054549 9532 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:53:59.059079 9590 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:59.059257 9590 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:59.059664 9590 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:59.060942 6481 ts_manager.cc:194] Re-registered known tserver with Master: 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411)
I20251212 08:53:59.063294 6481 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46148
I20251212 08:53:59.064966 9590 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
I20251212 08:53:59.065037 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:59.065268 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:59.065485 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:59.065735 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:59.066622 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:59.066751 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 14 LEADER]: Raft consensus shutting down.
I20251212 08:53:59.067087 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:59.081597 832 tablet_server.cc:195] TabletServer@127.0.208.1:35903 shutdown complete.
I20251212 08:53:59.085361 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:59.087875 9596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:59.087882 9595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:59.088447 9598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:59.088678 832 server_base.cc:1047] running on GCE node
I20251212 08:53:59.088966 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:59.089023 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:59.089047 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529639089044 us; error 0 us; skew 500 ppm
I20251212 08:53:59.089147 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:59.090385 832 webserver.cc:492] Webserver started at http://127.0.208.1:36883/ using document root <none> and password file <none>
I20251212 08:53:59.090562 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:59.090605 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:59.092491 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:59.093894 9603 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:59.094239 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:59.094349 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "8a42a66698e144409d7216b8f7b78219"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:59.094444 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:59.108202 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:59.108767 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:59.109412 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:59.115204 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:59.115299 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20251212 08:53:59.117446 9610 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:59.119094 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:59.119167 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20251212 08:53:59.134609 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35903
I20251212 08:53:59.134611 9667 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35903 every 8 connection(s)
I20251212 08:53:59.135511 9610 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=2 applied=25 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:59.136005 9610 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:59.136540 9610 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.019s user 0.015s sys 0.004s
I20251212 08:53:59.137312 9610 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:59.137552 9610 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:59.137812 9610 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 14.25, Last appended by leader: 25, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:59.142498 9610 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s
I20251212 08:53:59.142814 832 tablet_server.cc:178] TabletServer@127.0.208.2:42203 shutting down...
I20251212 08:53:59.143018 9668 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:59.143146 9668 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:59.143538 9668 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:59.142815 9610 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap starting.
I20251212 08:53:59.144820 6481 ts_manager.cc:194] Re-registered known tserver with Master: 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:53:59.147478 6481 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46164
I20251212 08:53:59.149643 9668 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:59.151960 9305 consensus_peers.cc:597] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 -> Peer 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903): Couldn't send request to peer 8a42a66698e144409d7216b8f7b78219. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20251212 08:53:59.156749 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:53:59.156977 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:59.157191 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 LEADER]: Raft consensus shutting down.
I20251212 08:53:59.157550 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:59.158228 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:53:59.158352 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Raft consensus shutting down.
I20251212 08:53:59.158491 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Raft consensus is shut down!
I20251212 08:53:59.173560 832 tablet_server.cc:195] TabletServer@127.0.208.2:42203 shutdown complete.
I20251212 08:53:59.177505 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:53:59.180051 9674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:59.180056 9675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:53:59.180891 9677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:53:59.181684 832 server_base.cc:1047] running on GCE node
I20251212 08:53:59.181972 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:53:59.182036 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:53:59.182121 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529639182117 us; error 0 us; skew 500 ppm
I20251212 08:53:59.182268 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:53:59.183624 832 webserver.cc:492] Webserver started at http://127.0.208.2:36849/ using document root <none> and password file <none>
I20251212 08:53:59.183828 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:53:59.183887 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:53:59.185906 832 fs_manager.cc:714] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20251212 08:53:59.187189 9610 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=1 applied=30 ignored=0} inserts{seen=17 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:59.187297 9682 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:53:59.187598 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:53:59.187657 9610 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Bootstrap complete.
I20251212 08:53:59.187695 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:53:59.187817 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:53:59.187948 9610 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent bootstrapping tablet: real 0.045s user 0.042s sys 0.000s
I20251212 08:53:59.194650 9610 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:59.194890 9610 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Initialized, Role: FOLLOWER
I20251212 08:53:59.195159 9610 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 30, Last appended: 14.30, Last appended by leader: 30, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:59.195783 9610 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219: Time spent starting tablet: real 0.008s user 0.002s sys 0.000s
I20251212 08:53:59.213093 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:53:59.213586 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:53:59.214066 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:53:59.220551 832 ts_tablet_manager.cc:585] Loaded tablet metadata (2 total tablets, 2 live tablets)
I20251212 08:53:59.220674 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20251212 08:53:59.223062 9689 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:59.224925 832 ts_tablet_manager.cc:616] Registered 2 tablets
I20251212 08:53:59.225018 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.004s sys 0.000s
I20251212 08:53:59.241670 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:42203
I20251212 08:53:59.241719 9746 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:42203 every 8 connection(s)
I20251212 08:53:59.246816 9689 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=25 overwritten=0 applied=25 ignored=0} inserts{seen=12 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:59.247345 9689 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:59.247630 9689 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.025s user 0.021s sys 0.000s
I20251212 08:53:59.248540 9689 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:59.248754 9689 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:59.249023 9689 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 25, Last appended: 14.25, Last appended by leader: 25, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:53:59.250538 9747 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:53:59.250708 9747 heartbeater.cc:461] Registering TS with master...
I20251212 08:53:59.250878 9689 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
W20251212 08:53:59.250988 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:59.251062 9747 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:53:59.251147 9689 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap starting.
I20251212 08:53:59.252507 6481 ts_manager.cc:194] Re-registered known tserver with Master: 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:53:59.254654 6481 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46180
I20251212 08:53:59.256801 9747 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:53:59.266932 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.279800 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:53:59.285601 9689 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=18 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:53:59.286095 9689 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Bootstrap complete.
I20251212 08:53:59.286362 9689 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent bootstrapping tablet: real 0.035s user 0.031s sys 0.000s
I20251212 08:53:59.287109 9689 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
W20251212 08:53:59.287177 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:53:59.287297 9689 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Initialized, Role: FOLLOWER
I20251212 08:53:59.287554 9689 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 14.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:53:59.288062 9689 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20251212 08:53:59.292814 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.302886 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.315572 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.325735 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.342216 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.357847 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.370904 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.383912 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.401516 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.417618 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.437867 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.459559 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.477707 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.499715 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.523586 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.545706 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.567852 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.594581 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.622725 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.647725 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.675269 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.703163 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.734335 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.764959 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.799134 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.834347 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.872321 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.906371 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:53:59.943492 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:53:59.983271 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.021459 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:54:00.062033 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.102983 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.144081 9545 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:34820: Illegal state: replica 808a87a7594d4fa0b5f8d1c78aebb8ed is not leader of this config: current role FOLLOWER
W20251212 08:54:00.186327 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:54:00.191603 9755 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:54:00.191776 9755 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:54:00.192751 9755 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:54:00.210610 9722 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 25 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:54:00.210610 9643 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 25 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:54:00.210857 9722 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 14.
I20251212 08:54:00.210884 9643 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 14.
I20251212 08:54:00.211329 9528 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:54:00.211603 9755 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Leader pre-election won for term 15
I20251212 08:54:00.211676 9755 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:54:00.211726 9755 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 14 FOLLOWER]: Advancing to term 15
I20251212 08:54:00.213954 9755 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:54:00.214576 9755 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 15 election: Requested vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:54:00.215035 9722 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 25 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:54:00.215040 9643 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" candidate_term: 15 candidate_status { last_received { term: 14 index: 25 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:54:00.215219 9722 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Advancing to term 15
I20251212 08:54:00.215246 9643 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Advancing to term 15
I20251212 08:54:00.217602 9722 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 15.
I20251212 08:54:00.217726 9643 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 808a87a7594d4fa0b5f8d1c78aebb8ed in term 15.
I20251212 08:54:00.218006 9528 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 808a87a7594d4fa0b5f8d1c78aebb8ed; no voters:
I20251212 08:54:00.218269 9755 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 15 FOLLOWER]: Leader election won for term 15
I20251212 08:54:00.218688 9755 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 15 LEADER]: Becoming Leader. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Running, Role: LEADER
I20251212 08:54:00.218940 9755 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 25, Committed index: 25, Last appended: 14.25, Last appended by leader: 25, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:54:00.221307 6481 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed reported cstate change: term changed from 14 to 15, leader changed from 8a42a66698e144409d7216b8f7b78219 (127.0.208.1) to 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4). New cstate: current_term: 15 leader_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } }
I20251212 08:54:00.234758 9722 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 FOLLOWER]: Refusing update from remote peer 808a87a7594d4fa0b5f8d1c78aebb8ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 25. Preceding OpId from leader: term: 15 index: 27. (index mismatch)
I20251212 08:54:00.234758 9643 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Refusing update from remote peer 808a87a7594d4fa0b5f8d1c78aebb8ed: Log matching property violated. Preceding OpId in replica: term: 14 index: 25. Preceding OpId from leader: term: 15 index: 27. (index mismatch)
I20251212 08:54:00.235483 9760 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251212 08:54:00.235792 9755 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 26, Last known committed idx: 25, Time since last communication: 0.000s
I20251212 08:54:00.242990 832 tablet_server.cc:178] TabletServer@127.0.208.3:39569 shutting down...
I20251212 08:54:00.255367 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:00.255684 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:54:00.255853 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 14 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:00.256074 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 14 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:00.271302 832 tablet_server.cc:195] TabletServer@127.0.208.3:39569 shutdown complete.
I20251212 08:54:00.275066 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:00.277558 9776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:00.277632 9777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:00.278041 9779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:00.278208 832 server_base.cc:1047] running on GCE node
I20251212 08:54:00.278519 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:00.278573 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:00.278599 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529640278596 us; error 0 us; skew 500 ppm
I20251212 08:54:00.278731 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:00.279919 832 webserver.cc:492] Webserver started at http://127.0.208.3:35191/ using document root <none> and password file <none>
I20251212 08:54:00.280119 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:00.280171 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:54:00.282025 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:00.283510 9784 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:00.283809 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:00.283895 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
format_stamp: "Formatted at 2025-12-12 08:53:31 on dist-test-slave-qdpb"
I20251212 08:54:00.284014 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:00.298895 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:00.299463 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:00.300042 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:54:00.301230 9791 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251212 08:54:00.303705 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:54:00.303791 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20251212 08:54:00.303884 832 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251212 08:54:00.305819 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:54:00.305879 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20251212 08:54:00.305964 9791 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap starting.
I20251212 08:54:00.321933 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39569
I20251212 08:54:00.321936 9848 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39569 every 8 connection(s)
W20251212 08:54:00.324455 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:54:00.330040 9849 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:54:00.330191 9849 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:00.330444 9791 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap replayed 1/1 log segments. Stats: ops{read=31 overwritten=0 applied=31 ignored=0} inserts{seen=18 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:54:00.330561 9849 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:54:00.331156 9791 tablet_bootstrap.cc:492] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Bootstrap complete.
W20251212 08:54:00.331187 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:54:00.331434 9791 ts_tablet_manager.cc:1403] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent bootstrapping tablet: real 0.026s user 0.024s sys 0.000s
I20251212 08:54:00.331856 6480 ts_manager.cc:194] Re-registered known tserver with Master: f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:54:00.332211 9791 raft_consensus.cc:359] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 14 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:54:00.332439 9791 raft_consensus.cc:740] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 14 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ceec6428fb4ee0b98c750a718d3a86, State: Initialized, Role: FOLLOWER
I20251212 08:54:00.332650 9791 consensus_queue.cc:260] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 31, Last appended: 14.31, Last appended by leader: 31, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:54:00.333123 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46182
I20251212 08:54:00.333317 9791 ts_tablet_manager.cc:1434] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:54:00.334326 9849 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:54:00.343464 9804 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38326: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.349611 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.359570 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.367508 9804 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38326: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.381146 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:54:00.388797 9767 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:54:00.389021 9767 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:54:00.389884 9767 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
W20251212 08:54:00.391978 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
I20251212 08:54:00.397114 9824 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 15 candidate_status { last_received { term: 14 index: 30 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:54:00.397258 9722 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 15 candidate_status { last_received { term: 14 index: 30 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:54:00.397449 9722 raft_consensus.cc:2410] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 for term 15 because replica has last-logged OpId of term: 14 index: 31, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 30.
I20251212 08:54:00.397449 9824 raft_consensus.cc:2410] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8a42a66698e144409d7216b8f7b78219 for term 15 because replica has last-logged OpId of term: 14 index: 31, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 30.
I20251212 08:54:00.397872 9607 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8a42a66698e144409d7216b8f7b78219; no voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86
I20251212 08:54:00.398209 9767 raft_consensus.cc:2749] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20251212 08:54:00.401741 9804 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38326: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.415069 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.427979 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.444276 9804 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38326: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:54:00.456758 9768 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:54:00.456979 9768 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:54:00.457937 9768 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
W20251212 08:54:00.464325 9702 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46192: Illegal state: replica 1b32dae091b44ce9a6fe4ae655cf01b9 is not leader of this config: current role FOLLOWER
I20251212 08:54:00.465962 9824 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 31 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" is_pre_election: true
I20251212 08:54:00.466187 9824 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 14.
I20251212 08:54:00.466522 9686 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:54:00.466784 9768 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20251212 08:54:00.466871 9768 raft_consensus.cc:493] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:54:00.466944 9768 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 14 FOLLOWER]: Advancing to term 15
I20251212 08:54:00.467587 9643 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 31 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:54:00.467808 9643 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 14.
I20251212 08:54:00.469683 9768 raft_consensus.cc:515] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:54:00.470209 9768 leader_election.cc:290] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 15 election: Requested vote from peers 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903), f2ceec6428fb4ee0b98c750a718d3a86 (127.0.208.3:39569)
I20251212 08:54:00.470527 9643 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 31 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219"
I20251212 08:54:00.470647 9824 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fd6fbc8b4b1d424cb5a077390cf3314d" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 15 candidate_status { last_received { term: 14 index: 31 } } ignore_live_leader: false dest_uuid: "f2ceec6428fb4ee0b98c750a718d3a86"
I20251212 08:54:00.470723 9643 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 14 FOLLOWER]: Advancing to term 15
I20251212 08:54:00.470827 9824 raft_consensus.cc:3060] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 14 FOLLOWER]: Advancing to term 15
I20251212 08:54:00.473012 9824 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 15.
I20251212 08:54:00.473405 9686 leader_election.cc:304] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, f2ceec6428fb4ee0b98c750a718d3a86; no voters:
I20251212 08:54:00.473672 9768 raft_consensus.cc:2804] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 FOLLOWER]: Leader election won for term 15
I20251212 08:54:00.474129 9768 raft_consensus.cc:697] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 LEADER]: Becoming Leader. State: Replica: 1b32dae091b44ce9a6fe4ae655cf01b9, State: Running, Role: LEADER
I20251212 08:54:00.474211 9643 raft_consensus.cc:2468] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in term 15.
I20251212 08:54:00.474411 9768 consensus_queue.cc:237] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 31, Committed index: 31, Last appended: 14.31, Last appended by leader: 31, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } }
I20251212 08:54:00.476740 6480 catalog_manager.cc:5654] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 reported cstate change: term changed from 14 to 15. New cstate: current_term: 15 leader_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 } health_report { overall_health: UNKNOWN } } }
W20251212 08:54:00.481210 9623 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58730: Illegal state: replica 8a42a66698e144409d7216b8f7b78219 is not leader of this config: current role FOLLOWER
W20251212 08:54:00.498099 9804 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:38326: Illegal state: replica f2ceec6428fb4ee0b98c750a718d3a86 is not leader of this config: current role FOLLOWER
I20251212 08:54:00.518494 9824 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 15 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 31. Preceding OpId from leader: term: 15 index: 33. (index mismatch)
I20251212 08:54:00.518573 9643 raft_consensus.cc:1275] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Refusing update from remote peer 1b32dae091b44ce9a6fe4ae655cf01b9: Log matching property violated. Preceding OpId in replica: term: 14 index: 30. Preceding OpId from leader: term: 15 index: 33. (index mismatch)
I20251212 08:54:00.519107 9860 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 30, Time since last communication: 0.000s
I20251212 08:54:00.519390 9768 consensus_queue.cc:1048] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ceec6428fb4ee0b98c750a718d3a86" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 32, Last known committed idx: 31, Time since last communication: 0.000s
I20251212 08:54:00.526315 832 tablet_server.cc:178] TabletServer@127.0.208.4:45411 shutting down...
I20251212 08:54:00.537155 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:00.537515 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:54:00.537672 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 15 LEADER]: Raft consensus shutting down.
I20251212 08:54:00.537983 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 15 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:00.542658 832 tablet_server.cc:195] TabletServer@127.0.208.4:45411 shutdown complete.
I20251212 08:54:00.546159 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:00.548625 9867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:00.548919 9868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:00.549309 832 server_base.cc:1047] running on GCE node
W20251212 08:54:00.549396 9870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:00.549690 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:00.549748 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:00.549798 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529640549793 us; error 0 us; skew 500 ppm
I20251212 08:54:00.549965 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:00.551322 832 webserver.cc:492] Webserver started at http://127.0.208.4:44337/ using document root <none> and password file <none>
I20251212 08:54:00.551519 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:00.551576 832 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251212 08:54:00.553459 832 fs_manager.cc:714] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:54:00.554904 9875 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:00.555280 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:00.555397 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
format_stamp: "Formatted at 2025-12-12 08:53:32 on dist-test-slave-qdpb"
I20251212 08:54:00.555506 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:00.567907 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:00.568542 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:00.569106 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:54:00.572918 832 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251212 08:54:00.573004 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.003s user 0.001s sys 0.000s
I20251212 08:54:00.575155 832 ts_tablet_manager.cc:616] Registered 1 tablets
I20251212 08:54:00.575246 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20251212 08:54:00.575300 9882 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap starting.
I20251212 08:54:00.592476 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.4:45411
I20251212 08:54:00.592494 9939 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.4:45411 every 8 connection(s)
I20251212 08:54:00.597569 9882 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap replayed 1/1 log segments. Stats: ops{read=27 overwritten=0 applied=27 ignored=0} inserts{seen=13 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251212 08:54:00.598237 9882 tablet_bootstrap.cc:492] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Bootstrap complete.
I20251212 08:54:00.598621 9882 ts_tablet_manager.cc:1403] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.004s
I20251212 08:54:00.599660 9882 raft_consensus.cc:359] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 15 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:54:00.599866 9882 raft_consensus.cc:740] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 15 FOLLOWER]: Becoming Follower/Learner. State: Replica: 808a87a7594d4fa0b5f8d1c78aebb8ed, State: Initialized, Role: FOLLOWER
I20251212 08:54:00.600148 9882 consensus_queue.cc:260] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 27, Last appended: 15.27, Last appended by leader: 27, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:54:00.600503 9940 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45467
I20251212 08:54:00.600632 9940 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:00.600950 9882 ts_tablet_manager.cc:1434] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20251212 08:54:00.601042 9940 heartbeater.cc:507] Master 127.0.208.62:45467 requested a full tablet report, sending...
I20251212 08:54:00.602095 6480 ts_manager.cc:194] Re-registered known tserver with Master: 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411)
I20251212 08:54:00.603247 6480 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46186
I20251212 08:54:00.604257 9940 heartbeater.cc:499] Master 127.0.208.62:45467 was elected leader, sending a full tablet report...
W20251212 08:54:01.614480 9894 tablet_service.cc:3074] error setting up scanner e5c66feb2adb4e58afa29dfa1923f48e with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: Timed out waiting for ts: P: 1765529640608969 usec, L: 1 to be safe (mode: NON-LEADER). Current safe time: P: 1765529640233499 usec, L: 0 Physical time difference: 0.375s: new_scan_request { tablet_id: "1231948ca74843e28105be29550fbdf8" read_mode: READ_YOUR_WRITES propagated_timestamp: 7231609407934337024 cache_blocks: true order_mode: UNORDERED row_format_flags: 0 authz_token { token_data: "\010\270\262\357\311\006\"3\n\005slave\022*\n 848c343b3b6a4b0c87337fd2b88d113c\020\001\030\001 \001(\001" signature: "\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t""\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "0092661a0dda4456b3d03029b3ced593"
I20251212 08:54:01.614838 9894 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Scan from 127.0.0.1:34832 (request call id 1) took 1001 ms. Trace:
I20251212 08:54:01.614954 9894 rpcz_store.cc:276] 1212 08:54:00.613363 (+ 0us) service_pool.cc:168] Inserting onto call queue
1212 08:54:00.613436 (+ 73us) service_pool.cc:225] Handling call
1212 08:54:00.613578 (+ 142us) tablet_service.cc:2884] Created scanner e5c66feb2adb4e58afa29dfa1923f48e for tablet 1231948ca74843e28105be29550fbdf8, query id is 0092661a0dda4456b3d03029b3ced593
1212 08:54:00.614166 (+ 588us) tablet_service.cc:3024] Creating iterator
1212 08:54:00.614191 (+ 25us) tablet_service.cc:3402] Waiting safe time to advance
1212 08:54:01.614450 (+1000259us) tablet_service.cc:3049] Iterator created
1212 08:54:01.614467 (+ 17us) tablet_service.cc:3071] Iterator init: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: Timed out waiting for ts: P: 1765529640608969 usec, L: 1 to be safe (mode: NON-LEADER). Current safe time: P: 1765529640233499 usec, L: 0 Physical time difference: 0.375s
1212 08:54:01.614822 (+ 355us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20251212 08:54:01.637246 9894 tablet_service.cc:3074] error setting up scanner d41c258c1a21480798ecc08830cb63e3 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 1.038s (maximum allowed 0.750s): new_scan_request { tablet_id: "1231948ca74843e28105be29550fbdf8" read_mode: READ_YOUR_WRITES propagated_timestamp: 7231609407934337024 cache_blocks: true order_mode: UNORDERED row_format_flags: 0 authz_token { token_data: "\010\270\262\357\311\006\"3\n\005slave\022*\n 848c343b3b6a4b0c87337fd2b88d113c\020\001\030\001 \001(\001" signature: "\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t""\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "0092661a0dda4456b3d03029b3ced593"
W20251212 08:54:01.660522 9894 tablet_service.cc:3074] error setting up scanner abcad11ae1fc4598bf8629d89ddf6deb with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 1.061s (maximum allowed 0.750s): new_scan_request { tablet_id: "1231948ca74843e28105be29550fbdf8" read_mode: READ_YOUR_WRITES propagated_timestamp: 7231609407934337024 cache_blocks: true order_mode: UNORDERED row_format_flags: 0 authz_token { token_data: "\010\270\262\357\311\006\"3\n\005slave\022*\n 848c343b3b6a4b0c87337fd2b88d113c\020\001\030\001 \001(\001" signature: "\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t""\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "0092661a0dda4456b3d03029b3ced593"
W20251212 08:54:01.707819 9894 tablet_service.cc:3074] error setting up scanner d062b650e52e48c39f5a0b49bff27772 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 1.109s (maximum allowed 0.750s): new_scan_request { tablet_id: "1231948ca74843e28105be29550fbdf8" read_mode: READ_YOUR_WRITES propagated_timestamp: 7231609407934337024 cache_blocks: true order_mode: UNORDERED row_format_flags: 0 authz_token { token_data: "\010\270\262\357\311\006\"3\n\005slave\022*\n 848c343b3b6a4b0c87337fd2b88d113c\020\001\030\001 \001(\001" signature: "\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t""\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "0092661a0dda4456b3d03029b3ced593"
I20251212 08:54:01.742266 9945 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:54:01.742458 9945 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:54:01.743222 9945 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:54:01.743551 9865 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:54:01.743783 9722 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 16 candidate_status { last_received { term: 15 index: 27 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" is_pre_election: true
I20251212 08:54:01.743733 9865 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:54:01.744237 9722 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 15.
I20251212 08:54:01.744696 9606 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:54:01.744961 9945 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20251212 08:54:01.745042 9945 raft_consensus.cc:493] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 808a87a7594d4fa0b5f8d1c78aebb8ed)
I20251212 08:54:01.745106 9945 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Advancing to term 16
I20251212 08:54:01.745850 9643 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 27 } } ignore_live_leader: false dest_uuid: "8a42a66698e144409d7216b8f7b78219" is_pre_election: true
I20251212 08:54:01.749228 9945 raft_consensus.cc:515] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:54:01.749714 9643 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 16: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:54:01.751850 9865 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 8a42a66698e144409d7216b8f7b78219 (127.0.208.1:35903)
I20251212 08:54:01.751888 9722 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 16 candidate_status { last_received { term: 15 index: 27 } } ignore_live_leader: false dest_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9"
I20251212 08:54:01.752033 9722 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 FOLLOWER]: Advancing to term 16
I20251212 08:54:01.756156 9722 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 16.
I20251212 08:54:01.756785 9606 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9, 8a42a66698e144409d7216b8f7b78219; no voters:
I20251212 08:54:01.757226 9915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 16 candidate_status { last_received { term: 15 index: 27 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:54:01.757441 9915 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 15.
I20251212 08:54:01.757933 9915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "8a42a66698e144409d7216b8f7b78219" candidate_term: 16 candidate_status { last_received { term: 15 index: 27 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed"
I20251212 08:54:01.758103 9915 raft_consensus.cc:3060] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 15 FOLLOWER]: Advancing to term 16
I20251212 08:54:01.758469 9950 raft_consensus.cc:2804] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 16 FOLLOWER]: Leader election won for term 16
I20251212 08:54:01.762949 9915 raft_consensus.cc:2468] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a42a66698e144409d7216b8f7b78219 in term 16.
I20251212 08:54:01.757923 9945 leader_election.cc:290] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [CANDIDATE]: Term 16 election: Requested vote from peers 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4:45411), 1b32dae091b44ce9a6fe4ae655cf01b9 (127.0.208.2:42203)
I20251212 08:54:01.764246 9950 raft_consensus.cc:697] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 16 LEADER]: Becoming Leader. State: Replica: 8a42a66698e144409d7216b8f7b78219, State: Running, Role: LEADER
I20251212 08:54:01.764600 9950 consensus_queue.cc:237] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 27, Committed index: 27, Last appended: 15.27, Last appended by leader: 27, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } }
I20251212 08:54:01.765002 9915 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1231948ca74843e28105be29550fbdf8" candidate_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" candidate_term: 16 candidate_status { last_received { term: 15 index: 27 } } ignore_live_leader: false dest_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" is_pre_election: true
I20251212 08:54:01.765233 9915 raft_consensus.cc:2393] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 1b32dae091b44ce9a6fe4ae655cf01b9 in current term 16: Already voted for candidate 8a42a66698e144409d7216b8f7b78219 in this term.
I20251212 08:54:01.765614 9684 leader_election.cc:304] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 1b32dae091b44ce9a6fe4ae655cf01b9; no voters: 808a87a7594d4fa0b5f8d1c78aebb8ed, 8a42a66698e144409d7216b8f7b78219
I20251212 08:54:01.765883 9865 raft_consensus.cc:2749] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20251212 08:54:01.768963 6480 catalog_manager.cc:5654] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 reported cstate change: term changed from 15 to 16, leader changed from 808a87a7594d4fa0b5f8d1c78aebb8ed (127.0.208.4) to 8a42a66698e144409d7216b8f7b78219 (127.0.208.1). New cstate: current_term: 16 leader_uuid: "8a42a66698e144409d7216b8f7b78219" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a42a66698e144409d7216b8f7b78219" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 } health_report { overall_health: UNKNOWN } } }
W20251212 08:54:01.814838 9894 tablet_service.cc:3074] error setting up scanner 893a4008e4c74470b0008367f13cf746 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 1.216s (maximum allowed 0.750s): new_scan_request { tablet_id: "1231948ca74843e28105be29550fbdf8" read_mode: READ_YOUR_WRITES propagated_timestamp: 7231609407934337024 cache_blocks: true order_mode: UNORDERED row_format_flags: 0 authz_token { token_data: "\010\270\262\357\311\006\"3\n\005slave\022*\n 848c343b3b6a4b0c87337fd2b88d113c\020\001\030\001 \001(\001" signature: "\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t""\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "0092661a0dda4456b3d03029b3ced593"
W20251212 08:54:02.105365 9894 tablet_service.cc:3074] error setting up scanner caf6dc60d77d497dab23f6f2febc9564 with request: Service unavailable: Timed out: could not wait for desired snapshot timestamp to be consistent: tablet hasn't heard from leader or there hasn't been a stable leader for 1.506s (maximum allowed 0.750s): new_scan_request { tablet_id: "1231948ca74843e28105be29550fbdf8" read_mode: READ_YOUR_WRITES propagated_timestamp: 7231609407934337024 cache_blocks: true order_mode: UNORDERED row_format_flags: 0 authz_token { token_data: "\010\270\262\357\311\006\"3\n\005slave\022*\n 848c343b3b6a4b0c87337fd2b88d113c\020\001\030\001 \001(\001" signature: "\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t""\201\366?\226\302\363\\\320y\232\002RIS>\220\017\354\310\247tq\271\342x)\357\345\372\214\3139\306\307\220\327\226[\312F4\321\362%\276Ut\365\211)B\362\250\254\273Lx\262\261;\334p\354t" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "0092661a0dda4456b3d03029b3ced593"
I20251212 08:54:02.189935 9722 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 16 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 15 index: 27. Preceding OpId from leader: term: 16 index: 28. (index mismatch)
I20251212 08:54:02.190624 9950 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b32dae091b44ce9a6fe4ae655cf01b9" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 42203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
I20251212 08:54:02.194459 9915 raft_consensus.cc:1275] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 16 FOLLOWER]: Refusing update from remote peer 8a42a66698e144409d7216b8f7b78219: Log matching property violated. Preceding OpId in replica: term: 15 index: 27. Preceding OpId from leader: term: 16 index: 28. (index mismatch)
I20251212 08:54:02.195060 9945 consensus_queue.cc:1048] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [LEADER]: Connected to new peer: Peer: permanent_uuid: "808a87a7594d4fa0b5f8d1c78aebb8ed" member_type: VOTER last_known_addr { host: "127.0.208.4" port: 45411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 28, Last known committed idx: 27, Time since last communication: 0.000s
W20251212 08:54:02.493175 832 scanner-internal.cc:458] Time spent opening tablet: real 1.880s user 0.002s sys 0.002s
I20251212 08:54:02.494153 832 tablet_server.cc:178] TabletServer@127.0.208.1:35903 shutting down...
I20251212 08:54:02.510059 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:02.510355 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:54:02.510535 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:02.510757 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 8a42a66698e144409d7216b8f7b78219 [term 15 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:02.511338 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:54:02.511440 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 16 LEADER]: Raft consensus shutting down.
I20251212 08:54:02.511690 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 8a42a66698e144409d7216b8f7b78219 [term 16 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:02.534842 832 tablet_server.cc:195] TabletServer@127.0.208.1:35903 shutdown complete.
I20251212 08:54:02.538354 832 tablet_server.cc:178] TabletServer@127.0.208.2:42203 shutting down...
I20251212 08:54:02.547870 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:02.548185 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:54:02.548429 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 LEADER]: Raft consensus shutting down.
I20251212 08:54:02.548779 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 15 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:02.549407 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:54:02.549542 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 16 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:02.549734 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 1b32dae091b44ce9a6fe4ae655cf01b9 [term 16 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:02.564152 832 tablet_server.cc:195] TabletServer@127.0.208.2:42203 shutdown complete.
I20251212 08:54:02.567689 832 tablet_server.cc:178] TabletServer@127.0.208.3:39569 shutting down...
I20251212 08:54:02.577831 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:02.578125 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:54:02.578310 832 raft_consensus.cc:2243] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 15 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:02.578541 832 raft_consensus.cc:2272] T fd6fbc8b4b1d424cb5a077390cf3314d P f2ceec6428fb4ee0b98c750a718d3a86 [term 15 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:02.591903 832 tablet_server.cc:195] TabletServer@127.0.208.3:39569 shutdown complete.
I20251212 08:54:02.595224 832 tablet_server.cc:178] TabletServer@127.0.208.4:45411 shutting down...
I20251212 08:54:02.607156 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:02.607463 832 tablet_replica.cc:333] stopping tablet replica
I20251212 08:54:02.607620 832 raft_consensus.cc:2243] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 16 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:02.607792 832 raft_consensus.cc:2272] T 1231948ca74843e28105be29550fbdf8 P 808a87a7594d4fa0b5f8d1c78aebb8ed [term 16 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:02.622040 832 tablet_server.cc:195] TabletServer@127.0.208.4:45411 shutdown complete.
I20251212 08:54:02.625411 832 master.cc:561] Master@127.0.208.62:45467 shutting down...
I20251212 08:54:02.634138 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:02.634390 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:02.634492 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P f5ed567a363645ad9cbdcea448ad3750: stopping tablet replica
W20251212 08:54:02.656396 832 rolling_log.cc:182] Unable to compress old log file: Not found: Unable to open input file to compress: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers.1765529557162900-832-0/client-test.dist-test-slave-qdpb.slave.diagnostics.20251212-085331.0.832: No such file or directory (error 2)
I20251212 08:54:02.656731 832 master.cc:583] Master@127.0.208.62:45467 shutdown complete.
[ OK ] MultiTServerClientTest.WriteWhileRestartingMultipleTabletServers (30976 ms)
[----------] 1 test from MultiTServerClientTest (30976 ms total)
[----------] 1 test from ClientTestAutoIncrementingColumn
[ RUN ] ClientTestAutoIncrementingColumn.AlterOperationNegatives
I20251212 08:54:02.670212 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:38405
I20251212 08:54:02.670781 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:02.673560 9964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:02.673609 9963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:02.673893 832 server_base.cc:1047] running on GCE node
W20251212 08:54:02.674053 9966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:02.674466 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:02.674535 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:02.674562 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529642674560 us; error 0 us; skew 500 ppm
I20251212 08:54:02.674731 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:02.675922 832 webserver.cc:492] Webserver started at http://127.0.208.62:33457/ using document root <none> and password file <none>
I20251212 08:54:02.676149 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:02.676211 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:02.676337 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:02.676978 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "c7635c2c367b42969464cecaf11f80fe"
format_stamp: "Formatted at 2025-12-12 08:54:02 on dist-test-slave-qdpb"
I20251212 08:54:02.679513 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:02.681252 9971 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.681588 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:02.681699 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "c7635c2c367b42969464cecaf11f80fe"
format_stamp: "Formatted at 2025-12-12 08:54:02 on dist-test-slave-qdpb"
I20251212 08:54:02.681823 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:02.723446 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:02.723996 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:02.739460 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:38405
I20251212 08:54:02.739504 10022 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:38405 every 8 connection(s)
I20251212 08:54:02.740871 10023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:02.745566 10023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe: Bootstrap starting.
I20251212 08:54:02.746806 10023 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:02.748347 10023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe: No bootstrap required, opened a new log
I20251212 08:54:02.748986 10023 raft_consensus.cc:359] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7635c2c367b42969464cecaf11f80fe" member_type: VOTER }
I20251212 08:54:02.749115 10023 raft_consensus.cc:385] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:02.749150 10023 raft_consensus.cc:740] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7635c2c367b42969464cecaf11f80fe, State: Initialized, Role: FOLLOWER
I20251212 08:54:02.749347 10023 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7635c2c367b42969464cecaf11f80fe" member_type: VOTER }
I20251212 08:54:02.749449 10023 raft_consensus.cc:399] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:02.749507 10023 raft_consensus.cc:493] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:02.749586 10023 raft_consensus.cc:3060] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:02.751617 10023 raft_consensus.cc:515] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7635c2c367b42969464cecaf11f80fe" member_type: VOTER }
I20251212 08:54:02.751801 10023 leader_election.cc:304] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7635c2c367b42969464cecaf11f80fe; no voters:
I20251212 08:54:02.752184 10023 leader_election.cc:290] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:02.752334 10026 raft_consensus.cc:2804] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:02.752763 10026 raft_consensus.cc:697] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 1 LEADER]: Becoming Leader. State: Replica: c7635c2c367b42969464cecaf11f80fe, State: Running, Role: LEADER
I20251212 08:54:02.753001 10023 sys_catalog.cc:565] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:02.752983 10026 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7635c2c367b42969464cecaf11f80fe" member_type: VOTER }
I20251212 08:54:02.753690 10028 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7635c2c367b42969464cecaf11f80fe. Latest consensus state: current_term: 1 leader_uuid: "c7635c2c367b42969464cecaf11f80fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7635c2c367b42969464cecaf11f80fe" member_type: VOTER } }
I20251212 08:54:02.753931 10028 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:02.753954 10027 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7635c2c367b42969464cecaf11f80fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7635c2c367b42969464cecaf11f80fe" member_type: VOTER } }
I20251212 08:54:02.754096 10027 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:02.755491 10031 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:02.757242 10031 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:02.758455 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:02.760782 10031 catalog_manager.cc:1357] Generated new cluster ID: 92c0ce035cfd4d17a9a8199ac1bee2e4
I20251212 08:54:02.760905 10031 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:02.770975 10031 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:02.771787 10031 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:02.780907 10031 catalog_manager.cc:6027] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe: Generated new TSK 0
I20251212 08:54:02.781181 10031 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:02.792098 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:02.794916 10044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:02.795094 10045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:02.795547 10047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:02.796500 832 server_base.cc:1047] running on GCE node
I20251212 08:54:02.796741 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:02.796801 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:02.796840 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529642796837 us; error 0 us; skew 500 ppm
I20251212 08:54:02.797009 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:02.798285 832 webserver.cc:492] Webserver started at http://127.0.208.1:36757/ using document root <none> and password file <none>
I20251212 08:54:02.798496 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:02.798568 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:02.798700 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:02.799350 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "895017f5189d477788457b97f5254538"
format_stamp: "Formatted at 2025-12-12 08:54:02 on dist-test-slave-qdpb"
I20251212 08:54:02.801983 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:02.803524 10052 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.803920 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:02.804028 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "895017f5189d477788457b97f5254538"
format_stamp: "Formatted at 2025-12-12 08:54:02 on dist-test-slave-qdpb"
I20251212 08:54:02.804122 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:02.816694 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:02.817193 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:02.817689 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:54:02.818471 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:02.818524 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.818579 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:02.818609 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.831682 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:46291
I20251212 08:54:02.831735 10114 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:46291 every 8 connection(s)
I20251212 08:54:02.833274 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:02.836077 10119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:02.836108 10120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:02.837306 10122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:02.837522 832 server_base.cc:1047] running on GCE node
I20251212 08:54:02.837780 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:02.837836 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:02.837858 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529642837855 us; error 0 us; skew 500 ppm
I20251212 08:54:02.837990 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:02.839193 832 webserver.cc:492] Webserver started at http://127.0.208.2:35231/ using document root <none> and password file <none>
I20251212 08:54:02.839390 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:02.839459 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:02.839578 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:02.840188 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-1-root/instance:
uuid: "acad3bed3ea247cfa6988bc68e8b85c0"
format_stamp: "Formatted at 2025-12-12 08:54:02 on dist-test-slave-qdpb"
I20251212 08:54:02.842559 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:54:02.843904 10115 heartbeater.cc:344] Connected to a master server at 127.0.208.62:38405
I20251212 08:54:02.844038 10115 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:02.844345 10127 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.844473 10115 heartbeater.cc:507] Master 127.0.208.62:38405 requested a full tablet report, sending...
I20251212 08:54:02.844712 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:54:02.844821 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "acad3bed3ea247cfa6988bc68e8b85c0"
format_stamp: "Formatted at 2025-12-12 08:54:02 on dist-test-slave-qdpb"
I20251212 08:54:02.844967 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:02.845422 9988 ts_manager.cc:194] Registered new tserver with Master: 895017f5189d477788457b97f5254538 (127.0.208.1:46291)
I20251212 08:54:02.846653 9988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57636
I20251212 08:54:02.855026 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:02.855482 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:02.855981 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:54:02.856845 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:02.856896 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.856972 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:02.857010 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.869227 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:36723
I20251212 08:54:02.869252 10189 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:36723 every 8 connection(s)
I20251212 08:54:02.870819 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:02.873711 10193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:02.873735 10194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:02.876849 832 server_base.cc:1047] running on GCE node
W20251212 08:54:02.877616 10196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:02.877643 10190 heartbeater.cc:344] Connected to a master server at 127.0.208.62:38405
I20251212 08:54:02.877794 10190 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:02.877986 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:02.878043 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:02.878072 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529642878067 us; error 0 us; skew 500 ppm
I20251212 08:54:02.878171 10190 heartbeater.cc:507] Master 127.0.208.62:38405 requested a full tablet report, sending...
I20251212 08:54:02.878245 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:02.879107 9988 ts_manager.cc:194] Registered new tserver with Master: acad3bed3ea247cfa6988bc68e8b85c0 (127.0.208.2:36723)
I20251212 08:54:02.879493 832 webserver.cc:492] Webserver started at http://127.0.208.3:39563/ using document root <none> and password file <none>
I20251212 08:54:02.879706 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:02.879778 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:02.879891 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:02.880155 9988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57638
I20251212 08:54:02.880649 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-2-root/instance:
uuid: "61b36f3d45604a46aa9b5c9cadb31ac0"
format_stamp: "Formatted at 2025-12-12 08:54:02 on dist-test-slave-qdpb"
I20251212 08:54:02.883140 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20251212 08:54:02.884670 10201 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.885011 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:02.885103 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "61b36f3d45604a46aa9b5c9cadb31ac0"
format_stamp: "Formatted at 2025-12-12 08:54:02 on dist-test-slave-qdpb"
I20251212 08:54:02.885210 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.ClientTestAutoIncrementingColumn.AlterOperationNegatives.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:02.902338 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:02.902940 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:02.903498 832 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251212 08:54:02.904330 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:02.904420 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.904522 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:02.904564 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.916103 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:35695
I20251212 08:54:02.916128 10263 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:35695 every 8 connection(s)
I20251212 08:54:02.922549 10264 heartbeater.cc:344] Connected to a master server at 127.0.208.62:38405
I20251212 08:54:02.922672 10264 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:02.923053 10264 heartbeater.cc:507] Master 127.0.208.62:38405 requested a full tablet report, sending...
I20251212 08:54:02.923820 9988 ts_manager.cc:194] Registered new tserver with Master: 61b36f3d45604a46aa9b5c9cadb31ac0 (127.0.208.3:35695)
I20251212 08:54:02.924573 9988 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57650
I20251212 08:54:02.924840 832 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.007836625s
I20251212 08:54:02.935757 9988 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57666:
name: "alter_operation_negatives_auto_incrementing_column"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "auto_incrementing_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
is_auto_incrementing: true
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 2
seed: 0
}
}
W20251212 08:54:02.936420 9988 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table alter_operation_negatives_auto_incrementing_column in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:54:02.957823 10229 tablet_service.cc:1505] Processing CreateTablet for tablet ffdd67bdb47a43b48a3089b641554100 (DEFAULT_TABLE table=alter_operation_negatives_auto_incrementing_column [id=9f67752ec24e459a8ab0559793b8917e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:02.958346 10229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffdd67bdb47a43b48a3089b641554100. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:02.957870 10228 tablet_service.cc:1505] Processing CreateTablet for tablet 1b1388eed4c345f5a83dae7f31ad6d92 (DEFAULT_TABLE table=alter_operation_negatives_auto_incrementing_column [id=9f67752ec24e459a8ab0559793b8917e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:02.960500 10228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1388eed4c345f5a83dae7f31ad6d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:02.963362 10155 tablet_service.cc:1505] Processing CreateTablet for tablet ffdd67bdb47a43b48a3089b641554100 (DEFAULT_TABLE table=alter_operation_negatives_auto_incrementing_column [id=9f67752ec24e459a8ab0559793b8917e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:02.963691 10155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffdd67bdb47a43b48a3089b641554100. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:02.967139 10279 tablet_bootstrap.cc:492] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0: Bootstrap starting.
I20251212 08:54:02.967748 10154 tablet_service.cc:1505] Processing CreateTablet for tablet 1b1388eed4c345f5a83dae7f31ad6d92 (DEFAULT_TABLE table=alter_operation_negatives_auto_incrementing_column [id=9f67752ec24e459a8ab0559793b8917e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:02.968216 10154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1388eed4c345f5a83dae7f31ad6d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:02.968712 10279 tablet.cc:502] Time spent fetching auto increment counter: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.969266 10279 tablet_bootstrap.cc:654] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:02.971160 10080 tablet_service.cc:1505] Processing CreateTablet for tablet ffdd67bdb47a43b48a3089b641554100 (DEFAULT_TABLE table=alter_operation_negatives_auto_incrementing_column [id=9f67752ec24e459a8ab0559793b8917e]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:02.971544 10080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ffdd67bdb47a43b48a3089b641554100. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:02.971462 10079 tablet_service.cc:1505] Processing CreateTablet for tablet 1b1388eed4c345f5a83dae7f31ad6d92 (DEFAULT_TABLE table=alter_operation_negatives_auto_incrementing_column [id=9f67752ec24e459a8ab0559793b8917e]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:02.972214 10079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1b1388eed4c345f5a83dae7f31ad6d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:02.977653 10279 tablet_bootstrap.cc:492] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0: No bootstrap required, opened a new log
I20251212 08:54:02.977873 10279 ts_tablet_manager.cc:1403] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.003s
I20251212 08:54:02.978631 10279 raft_consensus.cc:359] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } }
I20251212 08:54:02.978945 10279 raft_consensus.cc:385] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:02.979064 10279 raft_consensus.cc:740] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b36f3d45604a46aa9b5c9cadb31ac0, State: Initialized, Role: FOLLOWER
I20251212 08:54:02.979350 10279 consensus_queue.cc:260] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } }
I20251212 08:54:02.981415 10264 heartbeater.cc:499] Master 127.0.208.62:38405 was elected leader, sending a full tablet report...
I20251212 08:54:02.981491 10279 ts_tablet_manager.cc:1434] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20251212 08:54:02.981937 10282 tablet_bootstrap.cc:492] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538: Bootstrap starting.
I20251212 08:54:02.981683 10280 tablet_bootstrap.cc:492] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0: Bootstrap starting.
I20251212 08:54:02.983296 10282 tablet.cc:502] Time spent fetching auto increment counter: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.983515 10279 tablet_bootstrap.cc:492] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0: Bootstrap starting.
I20251212 08:54:02.983827 10282 tablet_bootstrap.cc:654] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:02.984504 10280 tablet.cc:502] Time spent fetching auto increment counter: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.984952 10279 tablet.cc:502] Time spent fetching auto increment counter: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.985119 10280 tablet_bootstrap.cc:654] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:02.985560 10279 tablet_bootstrap.cc:654] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:02.989442 10279 tablet_bootstrap.cc:492] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0: No bootstrap required, opened a new log
I20251212 08:54:02.989497 10280 tablet_bootstrap.cc:492] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0: No bootstrap required, opened a new log
I20251212 08:54:02.989522 10282 tablet_bootstrap.cc:492] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538: No bootstrap required, opened a new log
I20251212 08:54:02.989600 10279 ts_tablet_manager.cc:1403] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20251212 08:54:02.989631 10280 ts_tablet_manager.cc:1403] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20251212 08:54:02.989769 10282 ts_tablet_manager.cc:1403] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20251212 08:54:02.990401 10279 raft_consensus.cc:359] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } }
I20251212 08:54:02.990401 10280 raft_consensus.cc:359] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } }
I20251212 08:54:02.990446 10282 raft_consensus.cc:359] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } }
I20251212 08:54:02.990645 10279 raft_consensus.cc:385] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:02.990685 10282 raft_consensus.cc:385] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:02.990664 10280 raft_consensus.cc:385] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:02.990772 10282 raft_consensus.cc:740] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 895017f5189d477788457b97f5254538, State: Initialized, Role: FOLLOWER
I20251212 08:54:02.990829 10280 raft_consensus.cc:740] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acad3bed3ea247cfa6988bc68e8b85c0, State: Initialized, Role: FOLLOWER
I20251212 08:54:02.990744 10279 raft_consensus.cc:740] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61b36f3d45604a46aa9b5c9cadb31ac0, State: Initialized, Role: FOLLOWER
I20251212 08:54:02.991063 10279 consensus_queue.cc:260] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } }
I20251212 08:54:02.991063 10280 consensus_queue.cc:260] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } }
I20251212 08:54:02.991086 10282 consensus_queue.cc:260] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } }
I20251212 08:54:02.991642 10279 ts_tablet_manager.cc:1434] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20251212 08:54:02.991793 10280 ts_tablet_manager.cc:1434] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20251212 08:54:02.991802 10190 heartbeater.cc:499] Master 127.0.208.62:38405 was elected leader, sending a full tablet report...
I20251212 08:54:02.992055 10280 tablet_bootstrap.cc:492] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0: Bootstrap starting.
I20251212 08:54:02.994284 10280 tablet.cc:502] Time spent fetching auto increment counter: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:02.994872 10280 tablet_bootstrap.cc:654] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:02.998057 10280 tablet_bootstrap.cc:492] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0: No bootstrap required, opened a new log
I20251212 08:54:02.998183 10280 ts_tablet_manager.cc:1403] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251212 08:54:02.998859 10280 raft_consensus.cc:359] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } }
I20251212 08:54:02.999015 10280 raft_consensus.cc:385] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:02.999069 10280 raft_consensus.cc:740] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acad3bed3ea247cfa6988bc68e8b85c0, State: Initialized, Role: FOLLOWER
I20251212 08:54:02.999150 10115 heartbeater.cc:499] Master 127.0.208.62:38405 was elected leader, sending a full tablet report...
I20251212 08:54:02.999287 10280 consensus_queue.cc:260] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } }
I20251212 08:54:02.999642 10282 ts_tablet_manager.cc:1434] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538: Time spent starting tablet: real 0.010s user 0.004s sys 0.004s
I20251212 08:54:02.999936 10280 ts_tablet_manager.cc:1434] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:54:02.999953 10282 tablet_bootstrap.cc:492] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538: Bootstrap starting.
I20251212 08:54:03.001263 10282 tablet.cc:502] Time spent fetching auto increment counter: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:03.001847 10282 tablet_bootstrap.cc:654] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:03.003865 10282 tablet_bootstrap.cc:492] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538: No bootstrap required, opened a new log
I20251212 08:54:03.004027 10282 ts_tablet_manager.cc:1403] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:54:03.004745 10282 raft_consensus.cc:359] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } }
I20251212 08:54:03.004889 10282 raft_consensus.cc:385] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:03.004922 10282 raft_consensus.cc:740] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 895017f5189d477788457b97f5254538, State: Initialized, Role: FOLLOWER
I20251212 08:54:03.005129 10282 consensus_queue.cc:260] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } }
I20251212 08:54:03.005537 10282 ts_tablet_manager.cc:1434] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:03.020346 10288 raft_consensus.cc:493] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:54:03.020522 10288 raft_consensus.cc:515] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } }
I20251212 08:54:03.021191 10288 leader_election.cc:290] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 61b36f3d45604a46aa9b5c9cadb31ac0 (127.0.208.3:35695), acad3bed3ea247cfa6988bc68e8b85c0 (127.0.208.2:36723)
I20251212 08:54:03.027055 10239 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffdd67bdb47a43b48a3089b641554100" candidate_uuid: "895017f5189d477788457b97f5254538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" is_pre_election: true
I20251212 08:54:03.027304 10239 raft_consensus.cc:2468] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 895017f5189d477788457b97f5254538 in term 0.
I20251212 08:54:03.027328 10165 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffdd67bdb47a43b48a3089b641554100" candidate_uuid: "895017f5189d477788457b97f5254538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" is_pre_election: true
I20251212 08:54:03.027540 10165 raft_consensus.cc:2468] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 895017f5189d477788457b97f5254538 in term 0.
I20251212 08:54:03.027695 10054 leader_election.cc:304] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b36f3d45604a46aa9b5c9cadb31ac0, 895017f5189d477788457b97f5254538; no voters:
I20251212 08:54:03.027943 10288 raft_consensus.cc:2804] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:54:03.028029 10288 raft_consensus.cc:493] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:54:03.028081 10288 raft_consensus.cc:3060] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:03.030313 10288 raft_consensus.cc:515] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } }
I20251212 08:54:03.030789 10288 leader_election.cc:290] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [CANDIDATE]: Term 1 election: Requested vote from peers 61b36f3d45604a46aa9b5c9cadb31ac0 (127.0.208.3:35695), acad3bed3ea247cfa6988bc68e8b85c0 (127.0.208.2:36723)
I20251212 08:54:03.031159 10239 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffdd67bdb47a43b48a3089b641554100" candidate_uuid: "895017f5189d477788457b97f5254538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0"
I20251212 08:54:03.031217 10165 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ffdd67bdb47a43b48a3089b641554100" candidate_uuid: "895017f5189d477788457b97f5254538" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acad3bed3ea247cfa6988bc68e8b85c0"
I20251212 08:54:03.031320 10165 raft_consensus.cc:3060] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:03.031320 10239 raft_consensus.cc:3060] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:03.034278 10165 raft_consensus.cc:2468] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 895017f5189d477788457b97f5254538 in term 1.
I20251212 08:54:03.034278 10239 raft_consensus.cc:2468] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 895017f5189d477788457b97f5254538 in term 1.
I20251212 08:54:03.034701 10054 leader_election.cc:304] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 895017f5189d477788457b97f5254538, acad3bed3ea247cfa6988bc68e8b85c0; no voters:
I20251212 08:54:03.034991 10288 raft_consensus.cc:2804] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:03.035418 10288 raft_consensus.cc:697] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 1 LEADER]: Becoming Leader. State: Replica: 895017f5189d477788457b97f5254538, State: Running, Role: LEADER
I20251212 08:54:03.035652 10288 consensus_queue.cc:237] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } }
I20251212 08:54:03.038017 9988 catalog_manager.cc:5654] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 reported cstate change: term changed from 0 to 1, leader changed from <none> to 895017f5189d477788457b97f5254538 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "895017f5189d477788457b97f5254538" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } health_report { overall_health: HEALTHY } } }
W20251212 08:54:03.083011 10116 tablet.cc:2378] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251212 08:54:03.128940 10283 raft_consensus.cc:493] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:54:03.129172 10283 raft_consensus.cc:515] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } }
I20251212 08:54:03.130023 10283 leader_election.cc:290] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 895017f5189d477788457b97f5254538 (127.0.208.1:46291), acad3bed3ea247cfa6988bc68e8b85c0 (127.0.208.2:36723)
I20251212 08:54:03.135671 10165 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b1388eed4c345f5a83dae7f31ad6d92" candidate_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" is_pre_election: true
I20251212 08:54:03.135689 10090 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b1388eed4c345f5a83dae7f31ad6d92" candidate_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "895017f5189d477788457b97f5254538" is_pre_election: true
I20251212 08:54:03.135978 10165 raft_consensus.cc:2468] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b36f3d45604a46aa9b5c9cadb31ac0 in term 0.
I20251212 08:54:03.135980 10090 raft_consensus.cc:2468] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 61b36f3d45604a46aa9b5c9cadb31ac0 in term 0.
I20251212 08:54:03.136444 10203 leader_election.cc:304] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b36f3d45604a46aa9b5c9cadb31ac0, acad3bed3ea247cfa6988bc68e8b85c0; no voters:
I20251212 08:54:03.136726 10283 raft_consensus.cc:2804] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:54:03.136832 10283 raft_consensus.cc:493] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:54:03.136936 10283 raft_consensus.cc:3060] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:03.139010 10283 raft_consensus.cc:515] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } }
I20251212 08:54:03.139552 10283 leader_election.cc:290] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [CANDIDATE]: Term 1 election: Requested vote from peers 895017f5189d477788457b97f5254538 (127.0.208.1:46291), acad3bed3ea247cfa6988bc68e8b85c0 (127.0.208.2:36723)
I20251212 08:54:03.140004 10165 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b1388eed4c345f5a83dae7f31ad6d92" candidate_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acad3bed3ea247cfa6988bc68e8b85c0"
I20251212 08:54:03.140020 10090 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "1b1388eed4c345f5a83dae7f31ad6d92" candidate_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "895017f5189d477788457b97f5254538"
I20251212 08:54:03.140187 10165 raft_consensus.cc:3060] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:03.140240 10090 raft_consensus.cc:3060] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:03.142257 10165 raft_consensus.cc:2468] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b36f3d45604a46aa9b5c9cadb31ac0 in term 1.
I20251212 08:54:03.142293 10090 raft_consensus.cc:2468] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 61b36f3d45604a46aa9b5c9cadb31ac0 in term 1.
I20251212 08:54:03.142649 10203 leader_election.cc:304] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 61b36f3d45604a46aa9b5c9cadb31ac0, acad3bed3ea247cfa6988bc68e8b85c0; no voters:
I20251212 08:54:03.142925 10283 raft_consensus.cc:2804] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:03.143323 10283 raft_consensus.cc:697] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 1 LEADER]: Becoming Leader. State: Replica: 61b36f3d45604a46aa9b5c9cadb31ac0, State: Running, Role: LEADER
I20251212 08:54:03.143612 10283 consensus_queue.cc:237] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } }
I20251212 08:54:03.146024 9987 catalog_manager.cc:5654] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 61b36f3d45604a46aa9b5c9cadb31ac0 (127.0.208.3). New cstate: current_term: 1 leader_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "895017f5189d477788457b97f5254538" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 46291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acad3bed3ea247cfa6988bc68e8b85c0" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 36723 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "61b36f3d45604a46aa9b5c9cadb31ac0" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 35695 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:03.193102 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:03.201725 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:03.202209 832 tablet_replica.cc:333] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538: stopping tablet replica
I20251212 08:54:03.202463 832 raft_consensus.cc:2243] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:03.202638 832 raft_consensus.cc:2272] T 1b1388eed4c345f5a83dae7f31ad6d92 P 895017f5189d477788457b97f5254538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.203318 832 tablet_replica.cc:333] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538: stopping tablet replica
I20251212 08:54:03.203437 832 raft_consensus.cc:2243] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:03.203630 832 pending_rounds.cc:70] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538: Trying to abort 1 pending ops.
I20251212 08:54:03.203709 832 pending_rounds.cc:77] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231609417876017152 op_type: NO_OP noop_request { }
I20251212 08:54:03.203819 832 raft_consensus.cc:2889] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251212 08:54:03.203898 832 raft_consensus.cc:2272] T ffdd67bdb47a43b48a3089b641554100 P 895017f5189d477788457b97f5254538 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.218411 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:03.222169 832 tablet_server.cc:178] TabletServer@127.0.208.2:0 shutting down...
I20251212 08:54:03.230842 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:03.231287 832 tablet_replica.cc:333] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0: stopping tablet replica
I20251212 08:54:03.231493 832 raft_consensus.cc:2243] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:03.231704 832 raft_consensus.cc:2272] T 1b1388eed4c345f5a83dae7f31ad6d92 P acad3bed3ea247cfa6988bc68e8b85c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.232468 832 tablet_replica.cc:333] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0: stopping tablet replica
I20251212 08:54:03.232614 832 raft_consensus.cc:2243] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:03.232770 832 raft_consensus.cc:2272] T ffdd67bdb47a43b48a3089b641554100 P acad3bed3ea247cfa6988bc68e8b85c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.247032 832 tablet_server.cc:195] TabletServer@127.0.208.2:0 shutdown complete.
I20251212 08:54:03.260232 832 tablet_server.cc:178] TabletServer@127.0.208.3:0 shutting down...
I20251212 08:54:03.270138 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:03.270617 832 tablet_replica.cc:333] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0: stopping tablet replica
I20251212 08:54:03.270821 832 raft_consensus.cc:2243] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:03.270975 832 raft_consensus.cc:2272] T ffdd67bdb47a43b48a3089b641554100 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.271581 832 tablet_replica.cc:333] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0: stopping tablet replica
I20251212 08:54:03.271693 832 raft_consensus.cc:2243] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:03.271860 832 pending_rounds.cc:70] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0: Trying to abort 1 pending ops.
I20251212 08:54:03.271894 832 pending_rounds.cc:77] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7231609418318594048 op_type: NO_OP noop_request { }
I20251212 08:54:03.271981 832 raft_consensus.cc:2889] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20251212 08:54:03.272039 832 raft_consensus.cc:2272] T 1b1388eed4c345f5a83dae7f31ad6d92 P 61b36f3d45604a46aa9b5c9cadb31ac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.281639 832 tablet_server.cc:195] TabletServer@127.0.208.3:0 shutdown complete.
I20251212 08:54:03.286623 832 master.cc:561] Master@127.0.208.62:38405 shutting down...
I20251212 08:54:03.296592 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:03.296947 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.297082 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P c7635c2c367b42969464cecaf11f80fe: stopping tablet replica
I20251212 08:54:03.630334 832 master.cc:583] Master@127.0.208.62:38405 shutdown complete.
[ OK ] ClientTestAutoIncrementingColumn.AlterOperationNegatives (972 ms)
[----------] 1 test from ClientTestAutoIncrementingColumn (972 ms total)
[----------] 1 test from KeepAlivePeriodically/KeepAlivePeriodicallyTest
[ RUN ] KeepAlivePeriodically/KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers/0
I20251212 08:54:03.642882 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:45559
I20251212 08:54:03.643432 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:03.646220 10298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:03.646322 10297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:03.647431 10300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:03.652042 832 server_base.cc:1047] running on GCE node
I20251212 08:54:03.652379 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:03.652443 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:03.652472 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529643652469 us; error 0 us; skew 500 ppm
I20251212 08:54:03.652654 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:03.661363 832 webserver.cc:492] Webserver started at http://127.0.208.62:46261/ using document root <none> and password file <none>
I20251212 08:54:03.661616 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:03.661674 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:03.661918 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:03.662650 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "87b3abf586ee4e0297fd58ab72f7c068"
format_stamp: "Formatted at 2025-12-12 08:54:03 on dist-test-slave-qdpb"
I20251212 08:54:03.665499 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20251212 08:54:03.667167 10305 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:03.667532 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:03.667621 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "87b3abf586ee4e0297fd58ab72f7c068"
format_stamp: "Formatted at 2025-12-12 08:54:03 on dist-test-slave-qdpb"
I20251212 08:54:03.667752 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:03.715425 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:03.715973 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:03.728989 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:45559
I20251212 08:54:03.729014 10366 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:45559 every 8 connection(s)
I20251212 08:54:03.730355 10367 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:03.735100 10367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068: Bootstrap starting.
I20251212 08:54:03.736248 10367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:03.737766 10367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068: No bootstrap required, opened a new log
I20251212 08:54:03.738415 10367 raft_consensus.cc:359] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b3abf586ee4e0297fd58ab72f7c068" member_type: VOTER }
I20251212 08:54:03.738533 10367 raft_consensus.cc:385] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:03.738566 10367 raft_consensus.cc:740] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87b3abf586ee4e0297fd58ab72f7c068, State: Initialized, Role: FOLLOWER
I20251212 08:54:03.738757 10367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b3abf586ee4e0297fd58ab72f7c068" member_type: VOTER }
I20251212 08:54:03.738867 10367 raft_consensus.cc:399] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:03.738933 10367 raft_consensus.cc:493] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:03.739015 10367 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:03.741094 10367 raft_consensus.cc:515] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b3abf586ee4e0297fd58ab72f7c068" member_type: VOTER }
I20251212 08:54:03.741266 10367 leader_election.cc:304] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87b3abf586ee4e0297fd58ab72f7c068; no voters:
I20251212 08:54:03.741724 10367 leader_election.cc:290] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:03.741868 10370 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:03.742300 10370 raft_consensus.cc:697] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 1 LEADER]: Becoming Leader. State: Replica: 87b3abf586ee4e0297fd58ab72f7c068, State: Running, Role: LEADER
I20251212 08:54:03.742524 10370 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b3abf586ee4e0297fd58ab72f7c068" member_type: VOTER }
I20251212 08:54:03.742537 10367 sys_catalog.cc:565] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:03.743260 10372 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87b3abf586ee4e0297fd58ab72f7c068. Latest consensus state: current_term: 1 leader_uuid: "87b3abf586ee4e0297fd58ab72f7c068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b3abf586ee4e0297fd58ab72f7c068" member_type: VOTER } }
I20251212 08:54:03.743516 10372 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:03.743479 10371 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87b3abf586ee4e0297fd58ab72f7c068" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87b3abf586ee4e0297fd58ab72f7c068" member_type: VOTER } }
I20251212 08:54:03.743651 10371 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:03.745317 10375 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:03.747136 10375 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:03.747792 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:03.750108 10375 catalog_manager.cc:1357] Generated new cluster ID: c4086513fb8c4a778f542faff6f55aec
I20251212 08:54:03.750187 10375 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:03.757367 10375 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:03.758252 10375 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:03.765435 10375 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068: Generated new TSK 0
I20251212 08:54:03.765681 10375 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:03.780776 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:03.783746 10389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:03.783766 10388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:03.784549 10391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:03.784600 832 server_base.cc:1047] running on GCE node
I20251212 08:54:03.784925 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:03.784988 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:03.785027 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529643785022 us; error 0 us; skew 500 ppm
I20251212 08:54:03.785174 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:03.786423 832 webserver.cc:492] Webserver started at http://127.0.208.1:44995/ using document root <none> and password file <none>
I20251212 08:54:03.786661 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:03.786731 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:03.786860 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:03.787495 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "145c8fa2da764a2ab185862490972e6c"
format_stamp: "Formatted at 2025-12-12 08:54:03 on dist-test-slave-qdpb"
I20251212 08:54:03.790242 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:03.791946 10396 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:03.792335 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:54:03.792431 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "145c8fa2da764a2ab185862490972e6c"
format_stamp: "Formatted at 2025-12-12 08:54:03 on dist-test-slave-qdpb"
I20251212 08:54:03.792522 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:03.811021 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:03.811502 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:03.813303 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:03.813441 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:03.813594 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:03.813700 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:03.832414 10466 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:36957 every 8 connection(s)
I20251212 08:54:03.832391 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:36957
I20251212 08:54:03.834008 10467 heartbeater.cc:344] Connected to a master server at 127.0.208.62:45559
I20251212 08:54:03.834116 10467 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:03.834406 10467 heartbeater.cc:507] Master 127.0.208.62:45559 requested a full tablet report, sending...
I20251212 08:54:03.835238 10322 ts_manager.cc:194] Registered new tserver with Master: 145c8fa2da764a2ab185862490972e6c (127.0.208.1:36957)
I20251212 08:54:03.835722 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002317649s
I20251212 08:54:03.836566 10322 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57848
I20251212 08:54:03.846755 10322 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57866:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:03.848160 10467 heartbeater.cc:499] Master 127.0.208.62:45559 was elected leader, sending a full tablet report...
I20251212 08:54:03.860699 10427 tablet_service.cc:1505] Processing CreateTablet for tablet adf823bcf0fd40208a4eca4f7367eb79 (DEFAULT_TABLE table=client-testtb [id=9a89b92a9f5540c7a0c82f12cc092bd6]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:03.860687 10429 tablet_service.cc:1505] Processing CreateTablet for tablet 6a91ad18fe13460a815f128a23787387 (DEFAULT_TABLE table=client-testtb [id=9a89b92a9f5540c7a0c82f12cc092bd6]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:03.861290 10429 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6a91ad18fe13460a815f128a23787387. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:03.861495 10427 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet adf823bcf0fd40208a4eca4f7367eb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:03.866921 10477 tablet_bootstrap.cc:492] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c: Bootstrap starting.
I20251212 08:54:03.868221 10477 tablet_bootstrap.cc:654] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:03.869993 10477 tablet_bootstrap.cc:492] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c: No bootstrap required, opened a new log
I20251212 08:54:03.870110 10477 ts_tablet_manager.cc:1403] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:03.870659 10477 raft_consensus.cc:359] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } }
I20251212 08:54:03.870790 10477 raft_consensus.cc:385] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:03.870863 10477 raft_consensus.cc:740] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145c8fa2da764a2ab185862490972e6c, State: Initialized, Role: FOLLOWER
I20251212 08:54:03.871080 10477 consensus_queue.cc:260] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } }
I20251212 08:54:03.871227 10477 raft_consensus.cc:399] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:03.871712 10477 raft_consensus.cc:493] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:03.871804 10477 raft_consensus.cc:3060] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:03.874255 10477 raft_consensus.cc:515] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } }
I20251212 08:54:03.874436 10477 leader_election.cc:304] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 145c8fa2da764a2ab185862490972e6c; no voters:
I20251212 08:54:03.874802 10477 leader_election.cc:290] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:03.874934 10479 raft_consensus.cc:2804] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:03.875371 10477 ts_tablet_manager.cc:1434] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:54:03.875595 10479 raft_consensus.cc:697] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 1 LEADER]: Becoming Leader. State: Replica: 145c8fa2da764a2ab185862490972e6c, State: Running, Role: LEADER
I20251212 08:54:03.875655 10477 tablet_bootstrap.cc:492] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c: Bootstrap starting.
I20251212 08:54:03.875841 10479 consensus_queue.cc:237] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } }
I20251212 08:54:03.877226 10477 tablet_bootstrap.cc:654] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:03.878175 10321 catalog_manager.cc:5654] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 145c8fa2da764a2ab185862490972e6c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "145c8fa2da764a2ab185862490972e6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:03.878952 10477 tablet_bootstrap.cc:492] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c: No bootstrap required, opened a new log
I20251212 08:54:03.879143 10477 ts_tablet_manager.cc:1403] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:54:03.879910 10477 raft_consensus.cc:359] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } }
I20251212 08:54:03.880107 10477 raft_consensus.cc:385] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:03.880178 10477 raft_consensus.cc:740] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 145c8fa2da764a2ab185862490972e6c, State: Initialized, Role: FOLLOWER
I20251212 08:54:03.880388 10477 consensus_queue.cc:260] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } }
I20251212 08:54:03.880532 10477 raft_consensus.cc:399] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:03.880602 10477 raft_consensus.cc:493] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:03.880697 10477 raft_consensus.cc:3060] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:03.883229 10477 raft_consensus.cc:515] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } }
I20251212 08:54:03.883394 10477 leader_election.cc:304] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 145c8fa2da764a2ab185862490972e6c; no voters:
I20251212 08:54:03.883567 10477 leader_election.cc:290] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:03.883639 10479 raft_consensus.cc:2804] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:03.883805 10479 raft_consensus.cc:697] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 1 LEADER]: Becoming Leader. State: Replica: 145c8fa2da764a2ab185862490972e6c, State: Running, Role: LEADER
I20251212 08:54:03.883998 10477 ts_tablet_manager.cc:1434] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20251212 08:54:03.884016 10479 consensus_queue.cc:237] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } }
I20251212 08:54:03.885792 10321 catalog_manager.cc:5654] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c reported cstate change: term changed from 0 to 1, leader changed from <none> to 145c8fa2da764a2ab185862490972e6c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "145c8fa2da764a2ab185862490972e6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "145c8fa2da764a2ab185862490972e6c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36957 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:03.898535 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:03.917186 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:03.917655 832 tablet_replica.cc:333] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c: stopping tablet replica
I20251212 08:54:03.917893 832 raft_consensus.cc:2243] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:03.918103 832 raft_consensus.cc:2272] T 6a91ad18fe13460a815f128a23787387 P 145c8fa2da764a2ab185862490972e6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.919010 832 tablet_replica.cc:333] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c: stopping tablet replica
I20251212 08:54:03.919134 832 raft_consensus.cc:2243] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:03.919286 832 raft_consensus.cc:2272] T adf823bcf0fd40208a4eca4f7367eb79 P 145c8fa2da764a2ab185862490972e6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.924324 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:03.929488 832 master.cc:561] Master@127.0.208.62:45559 shutting down...
I20251212 08:54:03.939530 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:03.939807 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:03.940001 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 87b3abf586ee4e0297fd58ab72f7c068: stopping tablet replica
I20251212 08:54:03.954593 832 master.cc:583] Master@127.0.208.62:45559 shutdown complete.
I20251212 08:54:03.962335 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:36443
I20251212 08:54:03.963006 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:03.965631 10487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:03.966071 10486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:03.967285 10489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:03.967947 832 server_base.cc:1047] running on GCE node
I20251212 08:54:03.968214 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:03.968278 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:03.968354 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529643968349 us; error 0 us; skew 500 ppm
I20251212 08:54:03.968538 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:03.970011 832 webserver.cc:492] Webserver started at http://127.0.208.62:37953/ using document root <none> and password file <none>
I20251212 08:54:03.970263 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:03.970336 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:03.970469 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:03.971143 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "edc1f65b7a1549e9baf915d651c23329"
format_stamp: "Formatted at 2025-12-12 08:54:03 on dist-test-slave-qdpb"
I20251212 08:54:03.973898 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20251212 08:54:03.975733 10494 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:03.976213 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:03.976359 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "edc1f65b7a1549e9baf915d651c23329"
format_stamp: "Formatted at 2025-12-12 08:54:03 on dist-test-slave-qdpb"
I20251212 08:54:03.976480 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:03.990612 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:03.991178 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:04.005810 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:36443
I20251212 08:54:04.005838 10555 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:36443 every 8 connection(s)
I20251212 08:54:04.007099 10556 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.011797 10556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329: Bootstrap starting.
I20251212 08:54:04.013010 10556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.014412 10556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329: No bootstrap required, opened a new log
I20251212 08:54:04.015043 10556 raft_consensus.cc:359] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1f65b7a1549e9baf915d651c23329" member_type: VOTER }
I20251212 08:54:04.015174 10556 raft_consensus.cc:385] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.015209 10556 raft_consensus.cc:740] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edc1f65b7a1549e9baf915d651c23329, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.015359 10556 consensus_queue.cc:260] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1f65b7a1549e9baf915d651c23329" member_type: VOTER }
I20251212 08:54:04.015463 10556 raft_consensus.cc:399] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:04.015504 10556 raft_consensus.cc:493] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:04.015555 10556 raft_consensus.cc:3060] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.017670 10556 raft_consensus.cc:515] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1f65b7a1549e9baf915d651c23329" member_type: VOTER }
I20251212 08:54:04.017845 10556 leader_election.cc:304] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edc1f65b7a1549e9baf915d651c23329; no voters:
I20251212 08:54:04.018200 10556 leader_election.cc:290] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:04.018373 10559 raft_consensus.cc:2804] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:04.018788 10559 raft_consensus.cc:697] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 1 LEADER]: Becoming Leader. State: Replica: edc1f65b7a1549e9baf915d651c23329, State: Running, Role: LEADER
I20251212 08:54:04.018998 10556 sys_catalog.cc:565] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:04.019002 10559 consensus_queue.cc:237] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1f65b7a1549e9baf915d651c23329" member_type: VOTER }
I20251212 08:54:04.020119 10560 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edc1f65b7a1549e9baf915d651c23329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1f65b7a1549e9baf915d651c23329" member_type: VOTER } }
I20251212 08:54:04.020390 10561 sys_catalog.cc:455] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edc1f65b7a1549e9baf915d651c23329. Latest consensus state: current_term: 1 leader_uuid: "edc1f65b7a1549e9baf915d651c23329" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edc1f65b7a1549e9baf915d651c23329" member_type: VOTER } }
I20251212 08:54:04.020538 10561 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:04.020381 10560 sys_catalog.cc:458] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:04.021620 10566 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:04.023370 10566 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:04.023811 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:04.026264 10566 catalog_manager.cc:1357] Generated new cluster ID: eba4fcd47f8d49a081642ee4cfbbc805
I20251212 08:54:04.026328 10566 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:04.044996 10566 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:04.045831 10566 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:04.051797 10566 catalog_manager.cc:6027] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329: Generated new TSK 0
I20251212 08:54:04.052021 10566 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:04.056661 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:04.059449 10577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:04.059590 10578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:04.061118 832 server_base.cc:1047] running on GCE node
W20251212 08:54:04.061396 10580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:04.061703 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:04.061771 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:04.061817 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529644061815 us; error 0 us; skew 500 ppm
I20251212 08:54:04.061944 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:04.063252 832 webserver.cc:492] Webserver started at http://127.0.208.1:37797/ using document root <none> and password file <none>
I20251212 08:54:04.063457 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:04.063516 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:04.063627 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:04.064317 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "89fa170fee064b29a94cb83e3d1061bc"
format_stamp: "Formatted at 2025-12-12 08:54:04 on dist-test-slave-qdpb"
I20251212 08:54:04.067035 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:54:04.068869 10585 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:04.069281 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:04.069373 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "89fa170fee064b29a94cb83e3d1061bc"
format_stamp: "Formatted at 2025-12-12 08:54:04 on dist-test-slave-qdpb"
I20251212 08:54:04.069475 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:04.093714 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:04.094183 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:04.095837 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:04.095984 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:04.096145 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:04.096247 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:04.111902 10655 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:33179 every 8 connection(s)
I20251212 08:54:04.111898 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:33179
I20251212 08:54:04.113701 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251212 08:54:04.113896 10656 heartbeater.cc:344] Connected to a master server at 127.0.208.62:36443
I20251212 08:54:04.114012 10656 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:04.114297 10656 heartbeater.cc:507] Master 127.0.208.62:36443 requested a full tablet report, sending...
I20251212 08:54:04.115149 10511 ts_manager.cc:194] Registered new tserver with Master: 89fa170fee064b29a94cb83e3d1061bc (127.0.208.1:33179)
W20251212 08:54:04.116150 10658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:04.116174 10659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:04.116677 10511 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35140
W20251212 08:54:04.116988 10661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:04.117655 832 server_base.cc:1047] running on GCE node
I20251212 08:54:04.117879 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:04.117918 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:04.117944 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529644117941 us; error 0 us; skew 500 ppm
I20251212 08:54:04.118046 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:04.119213 832 webserver.cc:492] Webserver started at http://127.0.208.2:40655/ using document root <none> and password file <none>
I20251212 08:54:04.119410 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:04.119469 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:04.119567 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:04.120200 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-1-root/instance:
uuid: "d14e0270c4bd411d85cffde8ccc90190"
format_stamp: "Formatted at 2025-12-12 08:54:04 on dist-test-slave-qdpb"
I20251212 08:54:04.122786 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:54:04.124410 10666 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:04.124738 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:04.124837 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-1-root
uuid: "d14e0270c4bd411d85cffde8ccc90190"
format_stamp: "Formatted at 2025-12-12 08:54:04 on dist-test-slave-qdpb"
I20251212 08:54:04.124938 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:04.128041 10656 heartbeater.cc:499] Master 127.0.208.62:36443 was elected leader, sending a full tablet report...
I20251212 08:54:04.135241 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:04.135759 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:04.137434 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:04.137508 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:04.137588 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:04.137637 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:04.157315 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.2:33017
I20251212 08:54:04.157337 10735 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.2:33017 every 8 connection(s)
I20251212 08:54:04.158859 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251212 08:54:04.159029 10736 heartbeater.cc:344] Connected to a master server at 127.0.208.62:36443
I20251212 08:54:04.159144 10736 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:04.159475 10736 heartbeater.cc:507] Master 127.0.208.62:36443 requested a full tablet report, sending...
I20251212 08:54:04.160480 10510 ts_manager.cc:194] Registered new tserver with Master: d14e0270c4bd411d85cffde8ccc90190 (127.0.208.2:33017)
I20251212 08:54:04.161540 10510 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35156
W20251212 08:54:04.161794 10738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:04.161914 10739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:04.166095 10741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:04.169512 832 server_base.cc:1047] running on GCE node
I20251212 08:54:04.169742 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:04.169791 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:04.169814 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529644169811 us; error 0 us; skew 500 ppm
I20251212 08:54:04.169922 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:04.171094 832 webserver.cc:492] Webserver started at http://127.0.208.3:37661/ using document root <none> and password file <none>
I20251212 08:54:04.171267 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:04.171320 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:04.171403 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:04.171984 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-2-root/instance:
uuid: "bb1517e63b6c48a99c657fec90e6c4ce"
format_stamp: "Formatted at 2025-12-12 08:54:04 on dist-test-slave-qdpb"
I20251212 08:54:04.172936 10736 heartbeater.cc:499] Master 127.0.208.62:36443 was elected leader, sending a full tablet report...
I20251212 08:54:04.174810 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20251212 08:54:04.176605 10746 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:04.176970 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:54:04.177135 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-2-root
uuid: "bb1517e63b6c48a99c657fec90e6c4ce"
format_stamp: "Formatted at 2025-12-12 08:54:04 on dist-test-slave-qdpb"
I20251212 08:54:04.177243 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.KeepAlivePeriodically_KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers_0.1765529557162900-832-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:04.202621 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:04.203213 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:04.205286 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:04.205372 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:04.205480 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:04.205550 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:04.227665 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.3:39883
I20251212 08:54:04.227705 10815 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.3:39883 every 8 connection(s)
I20251212 08:54:04.229276 10816 heartbeater.cc:344] Connected to a master server at 127.0.208.62:36443
I20251212 08:54:04.229395 10816 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:04.229710 10816 heartbeater.cc:507] Master 127.0.208.62:36443 requested a full tablet report, sending...
I20251212 08:54:04.230465 10511 ts_manager.cc:194] Registered new tserver with Master: bb1517e63b6c48a99c657fec90e6c4ce (127.0.208.3:39883)
I20251212 08:54:04.231069 832 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.00244212s
I20251212 08:54:04.231211 10511 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:35164
I20251212 08:54:04.242252 10510 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35188:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
hash_schema {
columns {
name: "key"
}
num_buckets: 3
seed: 0
}
}
W20251212 08:54:04.243055 10510 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table client-testtb in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251212 08:54:04.243171 10816 heartbeater.cc:499] Master 127.0.208.62:36443 was elected leader, sending a full tablet report...
I20251212 08:54:04.260257 10616 tablet_service.cc:1505] Processing CreateTablet for tablet f640510c1ef74c7f89f79a77efd03ca3 (DEFAULT_TABLE table=client-testtb [id=a3bbd914b0144d49908cb62784abc3ec]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:04.260890 10616 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f640510c1ef74c7f89f79a77efd03ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.262740 10614 tablet_service.cc:1505] Processing CreateTablet for tablet be82aa2c6dab44e3b13748148d03503d (DEFAULT_TABLE table=client-testtb [id=a3bbd914b0144d49908cb62784abc3ec]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:04.263285 10614 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be82aa2c6dab44e3b13748148d03503d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.264864 10615 tablet_service.cc:1505] Processing CreateTablet for tablet a0b4d94e837b4128b9d34a66144f9798 (DEFAULT_TABLE table=client-testtb [id=a3bbd914b0144d49908cb62784abc3ec]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:04.265352 10615 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b4d94e837b4128b9d34a66144f9798. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.291318 10830 tablet_bootstrap.cc:492] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc: Bootstrap starting.
I20251212 08:54:04.293983 10830 tablet_bootstrap.cc:654] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.295559 10780 tablet_service.cc:1505] Processing CreateTablet for tablet f640510c1ef74c7f89f79a77efd03ca3 (DEFAULT_TABLE table=client-testtb [id=a3bbd914b0144d49908cb62784abc3ec]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:04.295970 10780 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f640510c1ef74c7f89f79a77efd03ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.296218 10830 tablet_bootstrap.cc:492] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc: No bootstrap required, opened a new log
I20251212 08:54:04.296386 10830 ts_tablet_manager.cc:1403] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20251212 08:54:04.297086 10830 raft_consensus.cc:359] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.297230 10830 raft_consensus.cc:385] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.297293 10830 raft_consensus.cc:740] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89fa170fee064b29a94cb83e3d1061bc, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.297520 10830 consensus_queue.cc:260] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.298259 10830 ts_tablet_manager.cc:1434] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251212 08:54:04.298570 10830 tablet_bootstrap.cc:492] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc: Bootstrap starting.
I20251212 08:54:04.299989 10830 tablet_bootstrap.cc:654] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.302650 10834 tablet_bootstrap.cc:492] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce: Bootstrap starting.
I20251212 08:54:04.303082 10694 tablet_service.cc:1505] Processing CreateTablet for tablet be82aa2c6dab44e3b13748148d03503d (DEFAULT_TABLE table=client-testtb [id=a3bbd914b0144d49908cb62784abc3ec]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:04.303079 10696 tablet_service.cc:1505] Processing CreateTablet for tablet f640510c1ef74c7f89f79a77efd03ca3 (DEFAULT_TABLE table=client-testtb [id=a3bbd914b0144d49908cb62784abc3ec]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:04.302650 10830 tablet_bootstrap.cc:492] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc: No bootstrap required, opened a new log
I20251212 08:54:04.303452 10830 ts_tablet_manager.cc:1403] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251212 08:54:04.303525 10694 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be82aa2c6dab44e3b13748148d03503d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.303397 10695 tablet_service.cc:1505] Processing CreateTablet for tablet a0b4d94e837b4128b9d34a66144f9798 (DEFAULT_TABLE table=client-testtb [id=a3bbd914b0144d49908cb62784abc3ec]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:04.303818 10695 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b4d94e837b4128b9d34a66144f9798. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.304042 10778 tablet_service.cc:1505] Processing CreateTablet for tablet a0b4d94e837b4128b9d34a66144f9798 (DEFAULT_TABLE table=client-testtb [id=a3bbd914b0144d49908cb62784abc3ec]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:04.304394 10778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b4d94e837b4128b9d34a66144f9798. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.304816 10830 raft_consensus.cc:359] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.306160 10830 raft_consensus.cc:385] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.306260 10830 raft_consensus.cc:740] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89fa170fee064b29a94cb83e3d1061bc, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.306514 10830 consensus_queue.cc:260] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.306964 10696 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f640510c1ef74c7f89f79a77efd03ca3. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.307646 10830 ts_tablet_manager.cc:1434] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20251212 08:54:04.307952 10830 tablet_bootstrap.cc:492] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc: Bootstrap starting.
I20251212 08:54:04.306082 10834 tablet_bootstrap.cc:654] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.310590 10830 tablet_bootstrap.cc:654] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.312687 10778 tablet_service.cc:1505] Processing CreateTablet for tablet be82aa2c6dab44e3b13748148d03503d (DEFAULT_TABLE table=client-testtb [id=a3bbd914b0144d49908cb62784abc3ec]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:04.313035 10778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be82aa2c6dab44e3b13748148d03503d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:04.317020 10836 tablet_bootstrap.cc:492] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190: Bootstrap starting.
I20251212 08:54:04.318188 10834 tablet_bootstrap.cc:492] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce: No bootstrap required, opened a new log
I20251212 08:54:04.318387 10834 ts_tablet_manager.cc:1403] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.002s
I20251212 08:54:04.318840 10836 tablet_bootstrap.cc:654] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.319288 10834 raft_consensus.cc:359] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.319568 10834 raft_consensus.cc:385] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.319697 10834 raft_consensus.cc:740] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1517e63b6c48a99c657fec90e6c4ce, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.320010 10834 consensus_queue.cc:260] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.321693 10830 tablet_bootstrap.cc:492] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc: No bootstrap required, opened a new log
I20251212 08:54:04.321858 10830 ts_tablet_manager.cc:1403] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20251212 08:54:04.322834 10830 raft_consensus.cc:359] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.323045 10830 raft_consensus.cc:385] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.323119 10830 raft_consensus.cc:740] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89fa170fee064b29a94cb83e3d1061bc, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.323372 10830 consensus_queue.cc:260] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.323987 10830 ts_tablet_manager.cc:1434] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20251212 08:54:04.324827 10836 tablet_bootstrap.cc:492] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190: No bootstrap required, opened a new log
I20251212 08:54:04.324975 10836 ts_tablet_manager.cc:1403] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20251212 08:54:04.325947 10836 raft_consensus.cc:359] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.326190 10836 raft_consensus.cc:385] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.326313 10836 raft_consensus.cc:740] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14e0270c4bd411d85cffde8ccc90190, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.326576 10836 consensus_queue.cc:260] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.327301 10834 ts_tablet_manager.cc:1434] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce: Time spent starting tablet: real 0.009s user 0.004s sys 0.004s
I20251212 08:54:04.327301 10836 ts_tablet_manager.cc:1434] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:04.327610 10834 tablet_bootstrap.cc:492] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce: Bootstrap starting.
I20251212 08:54:04.328594 10836 tablet_bootstrap.cc:492] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190: Bootstrap starting.
I20251212 08:54:04.329228 10834 tablet_bootstrap.cc:654] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.331202 10836 tablet_bootstrap.cc:654] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.336396 10836 tablet_bootstrap.cc:492] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190: No bootstrap required, opened a new log
I20251212 08:54:04.336531 10836 ts_tablet_manager.cc:1403] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20251212 08:54:04.336544 10834 tablet_bootstrap.cc:492] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce: No bootstrap required, opened a new log
I20251212 08:54:04.336732 10834 ts_tablet_manager.cc:1403] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.001s
I20251212 08:54:04.337188 10836 raft_consensus.cc:359] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.337325 10836 raft_consensus.cc:385] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.337423 10836 raft_consensus.cc:740] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14e0270c4bd411d85cffde8ccc90190, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.337316 10834 raft_consensus.cc:359] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.337515 10834 raft_consensus.cc:385] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.337582 10834 raft_consensus.cc:740] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1517e63b6c48a99c657fec90e6c4ce, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.337695 10836 consensus_queue.cc:260] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.338174 10836 ts_tablet_manager.cc:1434] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:54:04.338116 10834 consensus_queue.cc:260] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.338424 10836 tablet_bootstrap.cc:492] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190: Bootstrap starting.
I20251212 08:54:04.338557 10834 ts_tablet_manager.cc:1434] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251212 08:54:04.338792 10834 tablet_bootstrap.cc:492] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce: Bootstrap starting.
I20251212 08:54:04.340440 10836 tablet_bootstrap.cc:654] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.340590 10834 tablet_bootstrap.cc:654] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:04.343822 10836 tablet_bootstrap.cc:492] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190: No bootstrap required, opened a new log
I20251212 08:54:04.343950 10836 ts_tablet_manager.cc:1403] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20251212 08:54:04.343982 10834 tablet_bootstrap.cc:492] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce: No bootstrap required, opened a new log
I20251212 08:54:04.344096 10834 ts_tablet_manager.cc:1403] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251212 08:54:04.350231 10836 raft_consensus.cc:359] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.357878 10836 raft_consensus.cc:385] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.357795 10834 raft_consensus.cc:359] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.358033 10836 raft_consensus.cc:740] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14e0270c4bd411d85cffde8ccc90190, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.358093 10834 raft_consensus.cc:385] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:04.358179 10834 raft_consensus.cc:740] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb1517e63b6c48a99c657fec90e6c4ce, State: Initialized, Role: FOLLOWER
I20251212 08:54:04.358444 10834 consensus_queue.cc:260] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.358408 10836 consensus_queue.cc:260] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.358915 10834 ts_tablet_manager.cc:1434] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20251212 08:54:04.358952 10836 ts_tablet_manager.cc:1434] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190: Time spent starting tablet: real 0.015s user 0.002s sys 0.000s
I20251212 08:54:04.427110 10832 raft_consensus.cc:493] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:54:04.427290 10832 raft_consensus.cc:515] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.427970 10832 leader_election.cc:290] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d14e0270c4bd411d85cffde8ccc90190 (127.0.208.2:33017), bb1517e63b6c48a99c657fec90e6c4ce (127.0.208.3:39883)
I20251212 08:54:04.435037 10791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be82aa2c6dab44e3b13748148d03503d" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" is_pre_election: true
I20251212 08:54:04.435066 10711 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be82aa2c6dab44e3b13748148d03503d" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14e0270c4bd411d85cffde8ccc90190" is_pre_election: true
I20251212 08:54:04.435359 10711 raft_consensus.cc:2468] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 0.
I20251212 08:54:04.435379 10791 raft_consensus.cc:2468] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 0.
I20251212 08:54:04.435845 10588 leader_election.cc:304] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89fa170fee064b29a94cb83e3d1061bc, d14e0270c4bd411d85cffde8ccc90190; no voters:
I20251212 08:54:04.436117 10832 raft_consensus.cc:2804] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:54:04.436230 10832 raft_consensus.cc:493] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:54:04.436359 10832 raft_consensus.cc:3060] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.438606 10832 raft_consensus.cc:515] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.439224 10832 leader_election.cc:290] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 election: Requested vote from peers d14e0270c4bd411d85cffde8ccc90190 (127.0.208.2:33017), bb1517e63b6c48a99c657fec90e6c4ce (127.0.208.3:39883)
I20251212 08:54:04.439661 10711 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be82aa2c6dab44e3b13748148d03503d" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14e0270c4bd411d85cffde8ccc90190"
I20251212 08:54:04.439685 10791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "be82aa2c6dab44e3b13748148d03503d" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1517e63b6c48a99c657fec90e6c4ce"
I20251212 08:54:04.439834 10711 raft_consensus.cc:3060] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.439851 10791 raft_consensus.cc:3060] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.442058 10791 raft_consensus.cc:2468] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 1.
I20251212 08:54:04.442058 10711 raft_consensus.cc:2468] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 1.
I20251212 08:54:04.442478 10588 leader_election.cc:304] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 89fa170fee064b29a94cb83e3d1061bc, bb1517e63b6c48a99c657fec90e6c4ce, d14e0270c4bd411d85cffde8ccc90190; no voters:
I20251212 08:54:04.442776 10832 raft_consensus.cc:2804] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:04.443217 10832 raft_consensus.cc:697] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 1 LEADER]: Becoming Leader. State: Replica: 89fa170fee064b29a94cb83e3d1061bc, State: Running, Role: LEADER
I20251212 08:54:04.443495 10832 consensus_queue.cc:237] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.446590 10510 catalog_manager.cc:5654] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 89fa170fee064b29a94cb83e3d1061bc (127.0.208.1). New cstate: current_term: 1 leader_uuid: "89fa170fee064b29a94cb83e3d1061bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } health_report { overall_health: UNKNOWN } } }
I20251212 08:54:04.453892 10832 raft_consensus.cc:493] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:54:04.454082 10832 raft_consensus.cc:515] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.454769 10832 leader_election.cc:290] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d14e0270c4bd411d85cffde8ccc90190 (127.0.208.2:33017), bb1517e63b6c48a99c657fec90e6c4ce (127.0.208.3:39883)
I20251212 08:54:04.455235 10711 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b4d94e837b4128b9d34a66144f9798" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14e0270c4bd411d85cffde8ccc90190" is_pre_election: true
I20251212 08:54:04.455289 10791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b4d94e837b4128b9d34a66144f9798" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" is_pre_election: true
I20251212 08:54:04.455468 10711 raft_consensus.cc:2468] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 0.
I20251212 08:54:04.455487 10791 raft_consensus.cc:2468] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 0.
I20251212 08:54:04.455945 10587 leader_election.cc:304] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89fa170fee064b29a94cb83e3d1061bc, bb1517e63b6c48a99c657fec90e6c4ce; no voters:
I20251212 08:54:04.456297 10832 raft_consensus.cc:2804] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:54:04.456418 10832 raft_consensus.cc:493] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:54:04.456521 10832 raft_consensus.cc:3060] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.458995 10832 raft_consensus.cc:515] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.459646 10832 leader_election.cc:290] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 election: Requested vote from peers d14e0270c4bd411d85cffde8ccc90190 (127.0.208.2:33017), bb1517e63b6c48a99c657fec90e6c4ce (127.0.208.3:39883)
I20251212 08:54:04.459972 10711 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b4d94e837b4128b9d34a66144f9798" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14e0270c4bd411d85cffde8ccc90190"
I20251212 08:54:04.460124 10791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a0b4d94e837b4128b9d34a66144f9798" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1517e63b6c48a99c657fec90e6c4ce"
I20251212 08:54:04.460186 10711 raft_consensus.cc:3060] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.460338 10791 raft_consensus.cc:3060] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.462661 10711 raft_consensus.cc:2468] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 1.
I20251212 08:54:04.462656 10791 raft_consensus.cc:2468] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 1.
I20251212 08:54:04.463165 10588 leader_election.cc:304] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89fa170fee064b29a94cb83e3d1061bc, d14e0270c4bd411d85cffde8ccc90190; no voters:
I20251212 08:54:04.463506 10832 raft_consensus.cc:2804] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:04.463666 10832 raft_consensus.cc:697] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 1 LEADER]: Becoming Leader. State: Replica: 89fa170fee064b29a94cb83e3d1061bc, State: Running, Role: LEADER
I20251212 08:54:04.463954 10832 consensus_queue.cc:237] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.466447 10510 catalog_manager.cc:5654] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 89fa170fee064b29a94cb83e3d1061bc (127.0.208.1). New cstate: current_term: 1 leader_uuid: "89fa170fee064b29a94cb83e3d1061bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } health_report { overall_health: UNKNOWN } } }
I20251212 08:54:04.647046 10832 raft_consensus.cc:493] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251212 08:54:04.647343 10832 raft_consensus.cc:515] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.648469 10711 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f640510c1ef74c7f89f79a77efd03ca3" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14e0270c4bd411d85cffde8ccc90190" is_pre_election: true
I20251212 08:54:04.648962 10711 raft_consensus.cc:2468] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 0.
I20251212 08:54:04.648981 10832 leader_election.cc:290] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d14e0270c4bd411d85cffde8ccc90190 (127.0.208.2:33017), bb1517e63b6c48a99c657fec90e6c4ce (127.0.208.3:39883)
I20251212 08:54:04.649266 10791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f640510c1ef74c7f89f79a77efd03ca3" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" is_pre_election: true
I20251212 08:54:04.649354 10588 leader_election.cc:304] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89fa170fee064b29a94cb83e3d1061bc, d14e0270c4bd411d85cffde8ccc90190; no voters:
I20251212 08:54:04.649421 10791 raft_consensus.cc:2468] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 0.
I20251212 08:54:04.649631 10832 raft_consensus.cc:2804] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251212 08:54:04.649718 10832 raft_consensus.cc:493] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251212 08:54:04.649798 10832 raft_consensus.cc:3060] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.652087 10832 raft_consensus.cc:515] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.652725 10832 leader_election.cc:290] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 election: Requested vote from peers d14e0270c4bd411d85cffde8ccc90190 (127.0.208.2:33017), bb1517e63b6c48a99c657fec90e6c4ce (127.0.208.3:39883)
I20251212 08:54:04.653110 10711 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f640510c1ef74c7f89f79a77efd03ca3" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d14e0270c4bd411d85cffde8ccc90190"
I20251212 08:54:04.653132 10791 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "f640510c1ef74c7f89f79a77efd03ca3" candidate_uuid: "89fa170fee064b29a94cb83e3d1061bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bb1517e63b6c48a99c657fec90e6c4ce"
I20251212 08:54:04.653234 10711 raft_consensus.cc:3060] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.653262 10791 raft_consensus.cc:3060] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:04.655345 10791 raft_consensus.cc:2468] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 1.
I20251212 08:54:04.655366 10711 raft_consensus.cc:2468] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89fa170fee064b29a94cb83e3d1061bc in term 1.
I20251212 08:54:04.655856 10587 leader_election.cc:304] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89fa170fee064b29a94cb83e3d1061bc, bb1517e63b6c48a99c657fec90e6c4ce; no voters:
I20251212 08:54:04.656121 10832 raft_consensus.cc:2804] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:04.656270 10832 raft_consensus.cc:697] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 1 LEADER]: Becoming Leader. State: Replica: 89fa170fee064b29a94cb83e3d1061bc, State: Running, Role: LEADER
I20251212 08:54:04.656560 10832 consensus_queue.cc:237] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } }
I20251212 08:54:04.658993 10510 catalog_manager.cc:5654] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 89fa170fee064b29a94cb83e3d1061bc (127.0.208.1). New cstate: current_term: 1 leader_uuid: "89fa170fee064b29a94cb83e3d1061bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89fa170fee064b29a94cb83e3d1061bc" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 33179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 } health_report { overall_health: UNKNOWN } } }
I20251212 08:54:04.906832 10832 consensus_queue.cc:1048] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:54:04.928139 10791 raft_consensus.cc:1275] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Refusing update from remote peer 89fa170fee064b29a94cb83e3d1061bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:54:04.928812 10832 consensus_queue.cc:1048] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:54:04.946508 10790 raft_consensus.cc:1275] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Refusing update from remote peer 89fa170fee064b29a94cb83e3d1061bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:54:04.946817 10710 raft_consensus.cc:1275] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Refusing update from remote peer 89fa170fee064b29a94cb83e3d1061bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:54:04.947225 10832 consensus_queue.cc:1048] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:54:04.949342 10832 consensus_queue.cc:1048] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:54:04.956655 10789 raft_consensus.cc:1275] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Refusing update from remote peer 89fa170fee064b29a94cb83e3d1061bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:54:04.957310 10832 consensus_queue.cc:1048] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "bb1517e63b6c48a99c657fec90e6c4ce" member_type: VOTER last_known_addr { host: "127.0.208.3" port: 39883 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:54:04.964781 10710 raft_consensus.cc:1275] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Refusing update from remote peer 89fa170fee064b29a94cb83e3d1061bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251212 08:54:04.965487 10865 consensus_queue.cc:1048] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14e0270c4bd411d85cffde8ccc90190" member_type: VOTER last_known_addr { host: "127.0.208.2" port: 33017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251212 08:54:05.078631 10861 mvcc.cc:204] Tried to move back new op lower bound from 7231609425715081216 to 7231609424515403776. Current Snapshot: MvccSnapshot[applied={T|T < 7231609425715081216}]
I20251212 08:54:05.652643 10595 scanners.cc:441] Expiring scanner id: 5ce6867de2d24a068cfb18c7dfca908c, of tablet f640510c1ef74c7f89f79a77efd03ca3, after 511 ms of inactivity, which is > TTL (500 ms).
I20251212 08:54:07.143119 10606 tablet_service.cc:3173] Scan: Not found: Scanner 5ce6867de2d24a068cfb18c7dfca908c not found (it may have expired): call sequence id=1, remote={username='slave'} at 127.0.0.1:34426
I20251212 08:54:07.145134 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:07.156455 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:07.156800 832 tablet_replica.cc:333] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc: stopping tablet replica
I20251212 08:54:07.157008 832 raft_consensus.cc:2243] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:07.157310 832 raft_consensus.cc:2272] T f640510c1ef74c7f89f79a77efd03ca3 P 89fa170fee064b29a94cb83e3d1061bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.157958 832 tablet_replica.cc:333] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc: stopping tablet replica
I20251212 08:54:07.158073 832 raft_consensus.cc:2243] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:07.158457 832 raft_consensus.cc:2272] T be82aa2c6dab44e3b13748148d03503d P 89fa170fee064b29a94cb83e3d1061bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.159024 832 tablet_replica.cc:333] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc: stopping tablet replica
I20251212 08:54:07.159132 832 raft_consensus.cc:2243] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:07.159337 832 raft_consensus.cc:2272] T a0b4d94e837b4128b9d34a66144f9798 P 89fa170fee064b29a94cb83e3d1061bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.174221 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:07.178788 832 tablet_server.cc:178] TabletServer@127.0.208.2:0 shutting down...
I20251212 08:54:07.188764 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:07.189059 832 tablet_replica.cc:333] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190: stopping tablet replica
I20251212 08:54:07.189265 832 raft_consensus.cc:2243] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:07.189502 832 raft_consensus.cc:2272] T f640510c1ef74c7f89f79a77efd03ca3 P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.190239 832 tablet_replica.cc:333] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190: stopping tablet replica
I20251212 08:54:07.190389 832 raft_consensus.cc:2243] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:07.190582 832 raft_consensus.cc:2272] T be82aa2c6dab44e3b13748148d03503d P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.191169 832 tablet_replica.cc:333] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190: stopping tablet replica
I20251212 08:54:07.191272 832 raft_consensus.cc:2243] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:07.191438 832 raft_consensus.cc:2272] T a0b4d94e837b4128b9d34a66144f9798 P d14e0270c4bd411d85cffde8ccc90190 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.205477 832 tablet_server.cc:195] TabletServer@127.0.208.2:0 shutdown complete.
I20251212 08:54:07.209564 832 tablet_server.cc:178] TabletServer@127.0.208.3:0 shutting down...
I20251212 08:54:07.218509 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:07.218770 832 tablet_replica.cc:333] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce: stopping tablet replica
I20251212 08:54:07.218948 832 raft_consensus.cc:2243] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:07.219153 832 raft_consensus.cc:2272] T be82aa2c6dab44e3b13748148d03503d P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.219811 832 tablet_replica.cc:333] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce: stopping tablet replica
I20251212 08:54:07.219930 832 raft_consensus.cc:2243] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:07.220106 832 raft_consensus.cc:2272] T f640510c1ef74c7f89f79a77efd03ca3 P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.220674 832 tablet_replica.cc:333] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce: stopping tablet replica
I20251212 08:54:07.220788 832 raft_consensus.cc:2243] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20251212 08:54:07.220945 832 raft_consensus.cc:2272] T a0b4d94e837b4128b9d34a66144f9798 P bb1517e63b6c48a99c657fec90e6c4ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.234936 832 tablet_server.cc:195] TabletServer@127.0.208.3:0 shutdown complete.
I20251212 08:54:07.239380 832 master.cc:561] Master@127.0.208.62:36443 shutting down...
I20251212 08:54:07.257270 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:07.257544 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.257663 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P edc1f65b7a1549e9baf915d651c23329: stopping tablet replica
I20251212 08:54:07.271304 832 master.cc:583] Master@127.0.208.62:36443 shutdown complete.
[ OK ] KeepAlivePeriodically/KeepAlivePeriodicallyTest.TestScannerKeepAlivePeriodicallyCrossServers/0 (3642 ms)
[----------] 1 test from KeepAlivePeriodically/KeepAlivePeriodicallyTest (3642 ms total)
[----------] 1 test from Params/LatestObservedTimestampParamTest
[ RUN ] Params/LatestObservedTimestampParamTest.Test/0
I20251212 08:54:07.285084 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:41187
I20251212 08:54:07.285612 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:07.288316 10885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:07.288812 832 server_base.cc:1047] running on GCE node
W20251212 08:54:07.288699 10887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:07.288939 10884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:07.289351 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:07.289417 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:07.289464 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529647289460 us; error 0 us; skew 500 ppm
I20251212 08:54:07.289610 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:07.290792 832 webserver.cc:492] Webserver started at http://127.0.208.62:35983/ using document root <none> and password file <none>
I20251212 08:54:07.290987 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:07.291052 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:07.291177 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:07.291779 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "06ae39b5fd7d4d32bb33f43cc1795090"
format_stamp: "Formatted at 2025-12-12 08:54:07 on dist-test-slave-qdpb"
I20251212 08:54:07.294420 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:54:07.296082 10892 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:07.296535 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:54:07.296659 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "06ae39b5fd7d4d32bb33f43cc1795090"
format_stamp: "Formatted at 2025-12-12 08:54:07 on dist-test-slave-qdpb"
I20251212 08:54:07.296774 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:07.327046 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:07.327589 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:07.344669 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:41187
I20251212 08:54:07.344703 10953 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:41187 every 8 connection(s)
I20251212 08:54:07.346263 10954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:07.351102 10954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090: Bootstrap starting.
I20251212 08:54:07.352358 10954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:07.353922 10954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090: No bootstrap required, opened a new log
I20251212 08:54:07.354519 10954 raft_consensus.cc:359] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ae39b5fd7d4d32bb33f43cc1795090" member_type: VOTER }
I20251212 08:54:07.354660 10954 raft_consensus.cc:385] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:07.354712 10954 raft_consensus.cc:740] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06ae39b5fd7d4d32bb33f43cc1795090, State: Initialized, Role: FOLLOWER
I20251212 08:54:07.354902 10954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ae39b5fd7d4d32bb33f43cc1795090" member_type: VOTER }
I20251212 08:54:07.355002 10954 raft_consensus.cc:399] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:07.355039 10954 raft_consensus.cc:493] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:07.355118 10954 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:07.357278 10954 raft_consensus.cc:515] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ae39b5fd7d4d32bb33f43cc1795090" member_type: VOTER }
I20251212 08:54:07.357463 10954 leader_election.cc:304] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06ae39b5fd7d4d32bb33f43cc1795090; no voters:
I20251212 08:54:07.357916 10954 leader_election.cc:290] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:07.358058 10957 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:07.358520 10957 raft_consensus.cc:697] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 1 LEADER]: Becoming Leader. State: Replica: 06ae39b5fd7d4d32bb33f43cc1795090, State: Running, Role: LEADER
I20251212 08:54:07.358771 10954 sys_catalog.cc:565] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:07.358750 10957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ae39b5fd7d4d32bb33f43cc1795090" member_type: VOTER }
I20251212 08:54:07.359652 10959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06ae39b5fd7d4d32bb33f43cc1795090. Latest consensus state: current_term: 1 leader_uuid: "06ae39b5fd7d4d32bb33f43cc1795090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ae39b5fd7d4d32bb33f43cc1795090" member_type: VOTER } }
I20251212 08:54:07.359622 10958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06ae39b5fd7d4d32bb33f43cc1795090" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06ae39b5fd7d4d32bb33f43cc1795090" member_type: VOTER } }
I20251212 08:54:07.359803 10959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:07.359819 10958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:07.360615 10962 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:07.361855 10962 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:07.363889 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:07.364678 10962 catalog_manager.cc:1357] Generated new cluster ID: 4b05d7cb8ade4a39a6d6213a0669b77f
I20251212 08:54:07.364756 10962 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:07.375948 10962 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:07.376837 10962 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:07.386682 10962 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090: Generated new TSK 0
I20251212 08:54:07.386934 10962 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:07.396895 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:07.399631 10976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:07.399644 10975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:07.399827 10978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:07.400153 832 server_base.cc:1047] running on GCE node
I20251212 08:54:07.400404 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:07.400456 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:07.400478 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529647400476 us; error 0 us; skew 500 ppm
I20251212 08:54:07.400599 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:07.401629 832 webserver.cc:492] Webserver started at http://127.0.208.1:36297/ using document root <none> and password file <none>
I20251212 08:54:07.401798 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:07.401854 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:07.401942 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:07.402513 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "2c95479bc4884d6fa1faa51677c57ba8"
format_stamp: "Formatted at 2025-12-12 08:54:07 on dist-test-slave-qdpb"
I20251212 08:54:07.404958 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20251212 08:54:07.406533 10983 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:07.406960 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:07.407058 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "2c95479bc4884d6fa1faa51677c57ba8"
format_stamp: "Formatted at 2025-12-12 08:54:07 on dist-test-slave-qdpb"
I20251212 08:54:07.407176 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_LatestObservedTimestampParamTest.Test_0.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:07.427548 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:07.428081 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:07.429710 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:07.429787 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:07.429906 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:07.429996 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:07.447199 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:36689
I20251212 08:54:07.447234 11053 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:36689 every 8 connection(s)
I20251212 08:54:07.448618 11054 heartbeater.cc:344] Connected to a master server at 127.0.208.62:41187
I20251212 08:54:07.448732 11054 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:07.449021 11054 heartbeater.cc:507] Master 127.0.208.62:41187 requested a full tablet report, sending...
I20251212 08:54:07.449760 10909 ts_manager.cc:194] Registered new tserver with Master: 2c95479bc4884d6fa1faa51677c57ba8 (127.0.208.1:36689)
I20251212 08:54:07.450373 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002310196s
I20251212 08:54:07.450717 10909 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:55972
I20251212 08:54:07.460484 10909 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55984:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:07.461848 11054 heartbeater.cc:499] Master 127.0.208.62:41187 was elected leader, sending a full tablet report...
I20251212 08:54:07.472918 11013 tablet_service.cc:1505] Processing CreateTablet for tablet c9bf6723b1a04b12b73d71883e5ccbc2 (DEFAULT_TABLE table=client-testtb [id=d45e31b2be454ea6b5d68a41bfae5e35]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:07.472918 11015 tablet_service.cc:1505] Processing CreateTablet for tablet e66117fa1b2e4da1aa32147620f85190 (DEFAULT_TABLE table=client-testtb [id=d45e31b2be454ea6b5d68a41bfae5e35]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:07.473480 11013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c9bf6723b1a04b12b73d71883e5ccbc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:07.473713 11015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e66117fa1b2e4da1aa32147620f85190. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:07.481140 11064 tablet_bootstrap.cc:492] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8: Bootstrap starting.
I20251212 08:54:07.482424 11064 tablet_bootstrap.cc:654] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:07.483906 11064 tablet_bootstrap.cc:492] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8: No bootstrap required, opened a new log
I20251212 08:54:07.484040 11064 ts_tablet_manager.cc:1403] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:07.484675 11064 raft_consensus.cc:359] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } }
I20251212 08:54:07.484809 11064 raft_consensus.cc:385] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:07.484862 11064 raft_consensus.cc:740] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c95479bc4884d6fa1faa51677c57ba8, State: Initialized, Role: FOLLOWER
I20251212 08:54:07.485072 11064 consensus_queue.cc:260] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } }
I20251212 08:54:07.485214 11064 raft_consensus.cc:399] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:07.485280 11064 raft_consensus.cc:493] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:07.485379 11064 raft_consensus.cc:3060] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:07.487824 11064 raft_consensus.cc:515] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } }
I20251212 08:54:07.488016 11064 leader_election.cc:304] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c95479bc4884d6fa1faa51677c57ba8; no voters:
I20251212 08:54:07.488628 11066 raft_consensus.cc:2804] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:07.488648 11064 leader_election.cc:290] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:07.489174 11066 raft_consensus.cc:697] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c95479bc4884d6fa1faa51677c57ba8, State: Running, Role: LEADER
I20251212 08:54:07.489483 11066 consensus_queue.cc:237] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } }
I20251212 08:54:07.489579 11064 ts_tablet_manager.cc:1434] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8: Time spent starting tablet: real 0.005s user 0.002s sys 0.004s
I20251212 08:54:07.489832 11064 tablet_bootstrap.cc:492] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8: Bootstrap starting.
I20251212 08:54:07.491626 11064 tablet_bootstrap.cc:654] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:07.491528 10908 catalog_manager.cc:5654] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c95479bc4884d6fa1faa51677c57ba8 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "2c95479bc4884d6fa1faa51677c57ba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:07.493505 11064 tablet_bootstrap.cc:492] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8: No bootstrap required, opened a new log
I20251212 08:54:07.493623 11064 ts_tablet_manager.cc:1403] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:54:07.494261 11064 raft_consensus.cc:359] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } }
I20251212 08:54:07.494405 11064 raft_consensus.cc:385] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:07.494479 11064 raft_consensus.cc:740] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c95479bc4884d6fa1faa51677c57ba8, State: Initialized, Role: FOLLOWER
I20251212 08:54:07.494645 11064 consensus_queue.cc:260] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } }
I20251212 08:54:07.494740 11064 raft_consensus.cc:399] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:07.494776 11064 raft_consensus.cc:493] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:07.494833 11064 raft_consensus.cc:3060] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:07.497277 11064 raft_consensus.cc:515] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } }
I20251212 08:54:07.497475 11064 leader_election.cc:304] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c95479bc4884d6fa1faa51677c57ba8; no voters:
I20251212 08:54:07.497658 11064 leader_election.cc:290] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:07.497725 11066 raft_consensus.cc:2804] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:07.497875 11066 raft_consensus.cc:697] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 LEADER]: Becoming Leader. State: Replica: 2c95479bc4884d6fa1faa51677c57ba8, State: Running, Role: LEADER
I20251212 08:54:07.498034 11064 ts_tablet_manager.cc:1434] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:54:07.498113 11066 consensus_queue.cc:237] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } }
I20251212 08:54:07.500123 10908 catalog_manager.cc:5654] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c95479bc4884d6fa1faa51677c57ba8 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "2c95479bc4884d6fa1faa51677c57ba8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c95479bc4884d6fa1faa51677c57ba8" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36689 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:07.547917 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:07.564206 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:07.564630 832 tablet_replica.cc:333] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8: stopping tablet replica
I20251212 08:54:07.564864 832 raft_consensus.cc:2243] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:07.565135 832 raft_consensus.cc:2272] T e66117fa1b2e4da1aa32147620f85190 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.566208 832 tablet_replica.cc:333] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8: stopping tablet replica
I20251212 08:54:07.566354 832 raft_consensus.cc:2243] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:07.566493 832 raft_consensus.cc:2272] T c9bf6723b1a04b12b73d71883e5ccbc2 P 2c95479bc4884d6fa1faa51677c57ba8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.574703 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:07.589565 832 master.cc:561] Master@127.0.208.62:41187 shutting down...
I20251212 08:54:07.600464 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:07.600787 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:07.600991 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 06ae39b5fd7d4d32bb33f43cc1795090: stopping tablet replica
I20251212 08:54:07.605401 832 master.cc:583] Master@127.0.208.62:41187 shutdown complete.
[ OK ] Params/LatestObservedTimestampParamTest.Test/0 (331 ms)
[----------] 1 test from Params/LatestObservedTimestampParamTest (331 ms total)
[----------] 1 test from BinaryColEncodings/BinaryEncodingNullPredicatesTest
[ RUN ] BinaryColEncodings/BinaryEncodingNullPredicatesTest.TestBinaryEncodings/2
I20251212 08:54:07.616154 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:37271
I20251212 08:54:07.616761 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:07.619091 11081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:07.619175 11082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:07.619822 832 server_base.cc:1047] running on GCE node
W20251212 08:54:07.619843 11084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:07.620157 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:07.620227 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:07.620270 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529647620266 us; error 0 us; skew 500 ppm
I20251212 08:54:07.620456 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:07.621601 832 webserver.cc:492] Webserver started at http://127.0.208.62:43931/ using document root <none> and password file <none>
I20251212 08:54:07.621804 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:07.621881 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:07.622002 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:07.622663 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "cb0931eac16b4215ae5b74afaf4a468c"
format_stamp: "Formatted at 2025-12-12 08:54:07 on dist-test-slave-qdpb"
I20251212 08:54:07.625263 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:54:07.626725 11089 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:07.627082 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:54:07.627190 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "cb0931eac16b4215ae5b74afaf4a468c"
format_stamp: "Formatted at 2025-12-12 08:54:07 on dist-test-slave-qdpb"
I20251212 08:54:07.627296 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:07.647011 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:07.647516 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:07.660193 11150 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:37271 every 8 connection(s)
I20251212 08:54:07.660169 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:37271
I20251212 08:54:07.661578 11151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:07.666342 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c: Bootstrap starting.
I20251212 08:54:07.667477 11151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:07.669019 11151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c: No bootstrap required, opened a new log
I20251212 08:54:07.669601 11151 raft_consensus.cc:359] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb0931eac16b4215ae5b74afaf4a468c" member_type: VOTER }
I20251212 08:54:07.669721 11151 raft_consensus.cc:385] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:07.669754 11151 raft_consensus.cc:740] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0931eac16b4215ae5b74afaf4a468c, State: Initialized, Role: FOLLOWER
I20251212 08:54:07.669926 11151 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb0931eac16b4215ae5b74afaf4a468c" member_type: VOTER }
I20251212 08:54:07.670042 11151 raft_consensus.cc:399] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:07.670078 11151 raft_consensus.cc:493] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:07.670126 11151 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:07.672209 11151 raft_consensus.cc:515] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb0931eac16b4215ae5b74afaf4a468c" member_type: VOTER }
I20251212 08:54:07.672402 11151 leader_election.cc:304] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb0931eac16b4215ae5b74afaf4a468c; no voters:
I20251212 08:54:07.672770 11151 leader_election.cc:290] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:07.672907 11154 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:07.673359 11154 raft_consensus.cc:697] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 1 LEADER]: Becoming Leader. State: Replica: cb0931eac16b4215ae5b74afaf4a468c, State: Running, Role: LEADER
I20251212 08:54:07.673566 11151 sys_catalog.cc:565] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:07.673570 11154 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb0931eac16b4215ae5b74afaf4a468c" member_type: VOTER }
I20251212 08:54:07.674544 11156 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb0931eac16b4215ae5b74afaf4a468c. Latest consensus state: current_term: 1 leader_uuid: "cb0931eac16b4215ae5b74afaf4a468c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb0931eac16b4215ae5b74afaf4a468c" member_type: VOTER } }
I20251212 08:54:07.674510 11155 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb0931eac16b4215ae5b74afaf4a468c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb0931eac16b4215ae5b74afaf4a468c" member_type: VOTER } }
I20251212 08:54:07.674697 11156 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:07.675150 11155 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:07.675658 11159 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:07.677464 11159 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:07.678333 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:07.680501 11159 catalog_manager.cc:1357] Generated new cluster ID: 2f71f659e39043fab151cf07c9ba5f95
I20251212 08:54:07.680583 11159 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:07.698411 11159 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:07.699200 11159 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:07.707491 11159 catalog_manager.cc:6027] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c: Generated new TSK 0
I20251212 08:54:07.707724 11159 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:07.710949 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:07.713529 11172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:07.713701 11173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:07.714205 11175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:07.714545 832 server_base.cc:1047] running on GCE node
I20251212 08:54:07.714753 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:07.714807 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:07.714848 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529647714844 us; error 0 us; skew 500 ppm
I20251212 08:54:07.714998 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:07.716070 832 webserver.cc:492] Webserver started at http://127.0.208.1:38811/ using document root <none> and password file <none>
I20251212 08:54:07.716264 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:07.716380 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:07.716504 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:07.717104 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "055e001b5a434fee80438a40fb702e4c"
format_stamp: "Formatted at 2025-12-12 08:54:07 on dist-test-slave-qdpb"
I20251212 08:54:07.719571 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:07.721230 11180 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:07.721616 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:07.721720 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "055e001b5a434fee80438a40fb702e4c"
format_stamp: "Formatted at 2025-12-12 08:54:07 on dist-test-slave-qdpb"
I20251212 08:54:07.721829 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.BinaryColEncodings_BinaryEncodingNullPredicatesTest.TestBinaryEncodings_2.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:07.741537 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:07.742012 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:07.743501 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:07.743568 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:07.743656 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:07.743703 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:07.760531 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:35473
I20251212 08:54:07.760547 11250 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:35473 every 8 connection(s)
I20251212 08:54:07.762156 11251 heartbeater.cc:344] Connected to a master server at 127.0.208.62:37271
I20251212 08:54:07.762288 11251 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:07.762585 11251 heartbeater.cc:507] Master 127.0.208.62:37271 requested a full tablet report, sending...
I20251212 08:54:07.763772 11106 ts_manager.cc:194] Registered new tserver with Master: 055e001b5a434fee80438a40fb702e4c (127.0.208.1:35473)
I20251212 08:54:07.764396 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.001218858s
I20251212 08:54:07.764986 11106 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:44186
I20251212 08:54:07.774268 11106 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44206:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:07.776403 11251 heartbeater.cc:499] Master 127.0.208.62:37271 was elected leader, sending a full tablet report...
I20251212 08:54:07.787003 11210 tablet_service.cc:1505] Processing CreateTablet for tablet cc9fba718ddc49a383ea93ee9d6115b1 (DEFAULT_TABLE table=client-testtb [id=58e67665da574a139787afe6ee16364a]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:07.787062 11212 tablet_service.cc:1505] Processing CreateTablet for tablet b2e216ae758c4d07a6060eb6ad5128cc (DEFAULT_TABLE table=client-testtb [id=58e67665da574a139787afe6ee16364a]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:07.787421 11210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cc9fba718ddc49a383ea93ee9d6115b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:07.787617 11212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b2e216ae758c4d07a6060eb6ad5128cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:07.793391 11261 tablet_bootstrap.cc:492] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c: Bootstrap starting.
I20251212 08:54:07.794710 11261 tablet_bootstrap.cc:654] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:07.796178 11261 tablet_bootstrap.cc:492] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c: No bootstrap required, opened a new log
I20251212 08:54:07.796316 11261 ts_tablet_manager.cc:1403] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20251212 08:54:07.796916 11261 raft_consensus.cc:359] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.797047 11261 raft_consensus.cc:385] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:07.797080 11261 raft_consensus.cc:740] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055e001b5a434fee80438a40fb702e4c, State: Initialized, Role: FOLLOWER
I20251212 08:54:07.797267 11261 consensus_queue.cc:260] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.797379 11261 raft_consensus.cc:399] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:07.797437 11261 raft_consensus.cc:493] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:07.797503 11261 raft_consensus.cc:3060] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:07.799826 11261 raft_consensus.cc:515] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.800074 11261 leader_election.cc:304] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 055e001b5a434fee80438a40fb702e4c; no voters:
I20251212 08:54:07.800536 11261 leader_election.cc:290] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:07.800652 11263 raft_consensus.cc:2804] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:07.801151 11263 raft_consensus.cc:697] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 1 LEADER]: Becoming Leader. State: Replica: 055e001b5a434fee80438a40fb702e4c, State: Running, Role: LEADER
I20251212 08:54:07.801391 11263 consensus_queue.cc:237] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.802485 11261 ts_tablet_manager.cc:1434] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20251212 08:54:07.802740 11261 tablet_bootstrap.cc:492] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c: Bootstrap starting.
I20251212 08:54:07.803254 11106 catalog_manager.cc:5654] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 055e001b5a434fee80438a40fb702e4c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "055e001b5a434fee80438a40fb702e4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:07.804147 11261 tablet_bootstrap.cc:654] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:07.805876 11261 tablet_bootstrap.cc:492] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c: No bootstrap required, opened a new log
I20251212 08:54:07.806010 11261 ts_tablet_manager.cc:1403] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:07.806721 11261 raft_consensus.cc:359] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.806905 11261 raft_consensus.cc:385] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:07.806977 11261 raft_consensus.cc:740] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055e001b5a434fee80438a40fb702e4c, State: Initialized, Role: FOLLOWER
I20251212 08:54:07.807160 11261 consensus_queue.cc:260] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.807319 11261 raft_consensus.cc:399] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:07.807382 11261 raft_consensus.cc:493] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:07.807467 11261 raft_consensus.cc:3060] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:07.810034 11261 raft_consensus.cc:515] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.810263 11261 leader_election.cc:304] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 055e001b5a434fee80438a40fb702e4c; no voters:
I20251212 08:54:07.810482 11261 leader_election.cc:290] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:07.810541 11264 raft_consensus.cc:2804] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:07.810738 11264 raft_consensus.cc:697] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 1 LEADER]: Becoming Leader. State: Replica: 055e001b5a434fee80438a40fb702e4c, State: Running, Role: LEADER
I20251212 08:54:07.810889 11261 ts_tablet_manager.cc:1434] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:54:07.810952 11264 consensus_queue.cc:237] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.812832 11105 catalog_manager.cc:5654] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 055e001b5a434fee80438a40fb702e4c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "055e001b5a434fee80438a40fb702e4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:07.824823 11105 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44206:
name: "BinaryEncodingNullPredicatesTestTable"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: DICT_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:07.831133 11212 tablet_service.cc:1505] Processing CreateTablet for tablet b3a4b1bbb5ae4c328914dbfe9004c83a (DEFAULT_TABLE table=BinaryEncodingNullPredicatesTestTable [id=45a16825d8ed4233907257d36caffb26]), partition=RANGE (key) PARTITION UNBOUNDED
I20251212 08:54:07.831561 11212 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3a4b1bbb5ae4c328914dbfe9004c83a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:07.836417 11261 tablet_bootstrap.cc:492] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c: Bootstrap starting.
I20251212 08:54:07.837740 11261 tablet_bootstrap.cc:654] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:07.839162 11261 tablet_bootstrap.cc:492] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c: No bootstrap required, opened a new log
I20251212 08:54:07.839277 11261 ts_tablet_manager.cc:1403] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:07.839926 11261 raft_consensus.cc:359] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.840068 11261 raft_consensus.cc:385] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:07.840132 11261 raft_consensus.cc:740] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 055e001b5a434fee80438a40fb702e4c, State: Initialized, Role: FOLLOWER
I20251212 08:54:07.840395 11261 consensus_queue.cc:260] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.840551 11261 raft_consensus.cc:399] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:07.840636 11261 raft_consensus.cc:493] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:07.840715 11261 raft_consensus.cc:3060] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:07.842850 11261 raft_consensus.cc:515] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.843016 11261 leader_election.cc:304] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 055e001b5a434fee80438a40fb702e4c; no voters:
I20251212 08:54:07.843138 11261 leader_election.cc:290] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:07.843228 11264 raft_consensus.cc:2804] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:07.843398 11264 raft_consensus.cc:697] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 1 LEADER]: Becoming Leader. State: Replica: 055e001b5a434fee80438a40fb702e4c, State: Running, Role: LEADER
I20251212 08:54:07.843500 11261 ts_tablet_manager.cc:1434] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:54:07.843626 11264 consensus_queue.cc:237] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } }
I20251212 08:54:07.845537 11105 catalog_manager.cc:5654] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 055e001b5a434fee80438a40fb702e4c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "055e001b5a434fee80438a40fb702e4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "055e001b5a434fee80438a40fb702e4c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 35473 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:08.834702 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:08.845297 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:08.845623 832 tablet_replica.cc:333] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c: stopping tablet replica
I20251212 08:54:08.845840 832 raft_consensus.cc:2243] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:08.846055 832 raft_consensus.cc:2272] T b3a4b1bbb5ae4c328914dbfe9004c83a P 055e001b5a434fee80438a40fb702e4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:08.856027 832 tablet_replica.cc:333] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c: stopping tablet replica
I20251212 08:54:08.856232 832 raft_consensus.cc:2243] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:08.856477 832 raft_consensus.cc:2272] T b2e216ae758c4d07a6060eb6ad5128cc P 055e001b5a434fee80438a40fb702e4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:08.857014 832 tablet_replica.cc:333] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c: stopping tablet replica
I20251212 08:54:08.857144 832 raft_consensus.cc:2243] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:08.857273 832 raft_consensus.cc:2272] T cc9fba718ddc49a383ea93ee9d6115b1 P 055e001b5a434fee80438a40fb702e4c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:08.861394 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:08.866290 832 master.cc:561] Master@127.0.208.62:37271 shutting down...
I20251212 08:54:08.876199 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:08.876506 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:08.876641 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P cb0931eac16b4215ae5b74afaf4a468c: stopping tablet replica
I20251212 08:54:08.890193 832 master.cc:583] Master@127.0.208.62:37271 shutdown complete.
[ OK ] BinaryColEncodings/BinaryEncodingNullPredicatesTest.TestBinaryEncodings/2 (1287 ms)
[----------] 1 test from BinaryColEncodings/BinaryEncodingNullPredicatesTest (1288 ms total)
[----------] 2 tests from Params/ClientTestImmutableColumn
[ RUN ] Params/ClientTestImmutableColumn.TestUpdateIgnore/1
I20251212 08:54:08.906231 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:37861
I20251212 08:54:08.906841 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:08.909490 11279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:08.909619 11280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:08.910012 11282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:08.910482 832 server_base.cc:1047] running on GCE node
I20251212 08:54:08.910732 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:08.910794 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:08.910832 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529648910829 us; error 0 us; skew 500 ppm
I20251212 08:54:08.910991 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:08.912261 832 webserver.cc:492] Webserver started at http://127.0.208.62:43353/ using document root <none> and password file <none>
I20251212 08:54:08.912511 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:08.912586 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:08.912712 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:08.913486 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "8535f490f0024b439e159fd6e85451b3"
format_stamp: "Formatted at 2025-12-12 08:54:08 on dist-test-slave-qdpb"
I20251212 08:54:08.916635 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20251212 08:54:08.918605 11287 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:08.918979 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.001s
I20251212 08:54:08.919094 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "8535f490f0024b439e159fd6e85451b3"
format_stamp: "Formatted at 2025-12-12 08:54:08 on dist-test-slave-qdpb"
I20251212 08:54:08.919234 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:08.952013 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:08.952647 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:08.966629 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:37861
I20251212 08:54:08.966643 11348 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:37861 every 8 connection(s)
I20251212 08:54:08.968130 11349 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:08.972875 11349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3: Bootstrap starting.
I20251212 08:54:08.974006 11349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:08.975457 11349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3: No bootstrap required, opened a new log
I20251212 08:54:08.976029 11349 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535f490f0024b439e159fd6e85451b3" member_type: VOTER }
I20251212 08:54:08.976154 11349 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:08.976191 11349 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8535f490f0024b439e159fd6e85451b3, State: Initialized, Role: FOLLOWER
I20251212 08:54:08.976428 11349 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535f490f0024b439e159fd6e85451b3" member_type: VOTER }
I20251212 08:54:08.976533 11349 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:08.976589 11349 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:08.976661 11349 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:08.978645 11349 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535f490f0024b439e159fd6e85451b3" member_type: VOTER }
I20251212 08:54:08.978821 11349 leader_election.cc:304] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8535f490f0024b439e159fd6e85451b3; no voters:
I20251212 08:54:08.979270 11349 leader_election.cc:290] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:08.979393 11352 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:08.980305 11349 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:08.980533 11352 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 1 LEADER]: Becoming Leader. State: Replica: 8535f490f0024b439e159fd6e85451b3, State: Running, Role: LEADER
I20251212 08:54:08.980757 11352 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535f490f0024b439e159fd6e85451b3" member_type: VOTER }
I20251212 08:54:08.981626 11353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8535f490f0024b439e159fd6e85451b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535f490f0024b439e159fd6e85451b3" member_type: VOTER } }
I20251212 08:54:08.981765 11353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:08.981896 11354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8535f490f0024b439e159fd6e85451b3. Latest consensus state: current_term: 1 leader_uuid: "8535f490f0024b439e159fd6e85451b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8535f490f0024b439e159fd6e85451b3" member_type: VOTER } }
I20251212 08:54:08.982069 11354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:08.985798 11358 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:08.987828 11358 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:08.991791 11358 catalog_manager.cc:1357] Generated new cluster ID: 1a4ba199c1f54c5c88087759fd0df760
I20251212 08:54:08.991911 11358 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:08.997287 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:09.011834 11358 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:09.013007 11358 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:09.022841 11358 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3: Generated new TSK 0
I20251212 08:54:09.023103 11358 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:09.029902 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:09.032811 11370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.032824 11371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.034236 11373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:09.035934 832 server_base.cc:1047] running on GCE node
I20251212 08:54:09.036194 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:09.036258 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:09.036340 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529649036336 us; error 0 us; skew 500 ppm
I20251212 08:54:09.036525 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:09.037935 832 webserver.cc:492] Webserver started at http://127.0.208.1:39703/ using document root <none> and password file <none>
I20251212 08:54:09.038182 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:09.038261 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:09.038388 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:09.039167 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.042183 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20251212 08:54:09.044083 11378 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.044523 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:54:09.044626 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.044739 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpdateIgnore_1.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:09.066847 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:09.067449 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:09.075629 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:09.075723 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.075800 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:09.075842 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.100020 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:45749
I20251212 08:54:09.100056 11447 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:45749 every 8 connection(s)
I20251212 08:54:09.106016 11448 heartbeater.cc:344] Connected to a master server at 127.0.208.62:37861
I20251212 08:54:09.106261 11448 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:09.106734 11448 heartbeater.cc:507] Master 127.0.208.62:37861 requested a full tablet report, sending...
I20251212 08:54:09.107645 11304 ts_manager.cc:194] Registered new tserver with Master: 2ac1030eda0b48c68c3af1cbf6e62e6a (127.0.208.1:45749)
I20251212 08:54:09.109190 11304 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58998
I20251212 08:54:09.110776 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 7.5778e-05s
I20251212 08:54:09.120673 11448 heartbeater.cc:499] Master 127.0.208.62:37861 was elected leader, sending a full tablet report...
I20251212 08:54:09.122279 11304 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59012:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "imm_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: true
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:09.142168 11409 tablet_service.cc:1505] Processing CreateTablet for tablet f9243cf860d944328bebb0c8119ea044 (DEFAULT_TABLE table=client-testtb [id=dacdefbe96464795895a03fe48c5b11f]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:09.142168 11408 tablet_service.cc:1505] Processing CreateTablet for tablet c100346078fa4fefb389e2db238d3a66 (DEFAULT_TABLE table=client-testtb [id=dacdefbe96464795895a03fe48c5b11f]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:09.142745 11409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9243cf860d944328bebb0c8119ea044. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:09.142982 11408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c100346078fa4fefb389e2db238d3a66. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:09.148826 11459 tablet_bootstrap.cc:492] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a: Bootstrap starting.
I20251212 08:54:09.150106 11459 tablet_bootstrap.cc:654] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:09.151654 11459 tablet_bootstrap.cc:492] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a: No bootstrap required, opened a new log
I20251212 08:54:09.151789 11459 ts_tablet_manager.cc:1403] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:09.152477 11459 raft_consensus.cc:359] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } }
I20251212 08:54:09.152608 11459 raft_consensus.cc:385] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:09.152680 11459 raft_consensus.cc:740] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac1030eda0b48c68c3af1cbf6e62e6a, State: Initialized, Role: FOLLOWER
I20251212 08:54:09.152882 11459 consensus_queue.cc:260] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } }
I20251212 08:54:09.153007 11459 raft_consensus.cc:399] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:09.153069 11459 raft_consensus.cc:493] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:09.153146 11459 raft_consensus.cc:3060] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:09.155231 11459 raft_consensus.cc:515] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } }
I20251212 08:54:09.155427 11459 leader_election.cc:304] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ac1030eda0b48c68c3af1cbf6e62e6a; no voters:
I20251212 08:54:09.155918 11459 leader_election.cc:290] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:09.156308 11461 raft_consensus.cc:2804] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:09.156802 11461 raft_consensus.cc:697] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 LEADER]: Becoming Leader. State: Replica: 2ac1030eda0b48c68c3af1cbf6e62e6a, State: Running, Role: LEADER
I20251212 08:54:09.157055 11461 consensus_queue.cc:237] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } }
I20251212 08:54:09.157878 11459 ts_tablet_manager.cc:1434] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20251212 08:54:09.158143 11459 tablet_bootstrap.cc:492] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a: Bootstrap starting.
I20251212 08:54:09.159664 11459 tablet_bootstrap.cc:654] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:09.161765 11304 catalog_manager.cc:5654] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ac1030eda0b48c68c3af1cbf6e62e6a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:09.164150 11459 tablet_bootstrap.cc:492] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a: No bootstrap required, opened a new log
I20251212 08:54:09.164330 11459 ts_tablet_manager.cc:1403] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20251212 08:54:09.165076 11459 raft_consensus.cc:359] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } }
I20251212 08:54:09.165232 11459 raft_consensus.cc:385] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:09.165313 11459 raft_consensus.cc:740] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac1030eda0b48c68c3af1cbf6e62e6a, State: Initialized, Role: FOLLOWER
I20251212 08:54:09.165530 11459 consensus_queue.cc:260] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } }
I20251212 08:54:09.165683 11459 raft_consensus.cc:399] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:09.165751 11459 raft_consensus.cc:493] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:09.165830 11459 raft_consensus.cc:3060] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:09.172960 11459 raft_consensus.cc:515] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } }
I20251212 08:54:09.173174 11459 leader_election.cc:304] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ac1030eda0b48c68c3af1cbf6e62e6a; no voters:
I20251212 08:54:09.173475 11461 raft_consensus.cc:2804] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:09.173606 11461 raft_consensus.cc:697] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 LEADER]: Becoming Leader. State: Replica: 2ac1030eda0b48c68c3af1cbf6e62e6a, State: Running, Role: LEADER
I20251212 08:54:09.173820 11461 consensus_queue.cc:237] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } }
I20251212 08:54:09.174039 11459 leader_election.cc:290] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:09.174692 11459 ts_tablet_manager.cc:1434] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a: Time spent starting tablet: real 0.010s user 0.004s sys 0.001s
I20251212 08:54:09.179665 11304 catalog_manager.cc:5654] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ac1030eda0b48c68c3af1cbf6e62e6a (127.0.208.1). New cstate: current_term: 1 leader_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ac1030eda0b48c68c3af1cbf6e62e6a" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 45749 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:09.232877 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:09.244141 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:09.244695 832 tablet_replica.cc:333] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a: stopping tablet replica
I20251212 08:54:09.244887 832 raft_consensus.cc:2243] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:09.245086 832 raft_consensus.cc:2272] T c100346078fa4fefb389e2db238d3a66 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:09.253474 832 tablet_replica.cc:333] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a: stopping tablet replica
I20251212 08:54:09.253736 832 raft_consensus.cc:2243] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:09.254012 832 raft_consensus.cc:2272] T f9243cf860d944328bebb0c8119ea044 P 2ac1030eda0b48c68c3af1cbf6e62e6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:09.259588 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:09.264266 832 master.cc:561] Master@127.0.208.62:37861 shutting down...
I20251212 08:54:09.273867 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:09.274108 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:09.274258 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8535f490f0024b439e159fd6e85451b3: stopping tablet replica
I20251212 08:54:09.278561 832 master.cc:583] Master@127.0.208.62:37861 shutdown complete.
[ OK ] Params/ClientTestImmutableColumn.TestUpdateIgnore/1 (386 ms)
[ RUN ] Params/ClientTestImmutableColumn.TestUpsert/1
I20251212 08:54:09.292169 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:40957
I20251212 08:54:09.292827 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:09.295420 11471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.295728 11473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.295734 11470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:09.297483 832 server_base.cc:1047] running on GCE node
I20251212 08:54:09.297708 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:09.297762 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:09.297799 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529649297795 us; error 0 us; skew 500 ppm
I20251212 08:54:09.297950 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:09.299204 832 webserver.cc:492] Webserver started at http://127.0.208.62:34811/ using document root <none> and password file <none>
I20251212 08:54:09.299414 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:09.299485 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:09.299597 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:09.300360 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "855ecc3da30b49a6b506d27c78120813"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.303512 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20251212 08:54:09.305312 11478 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.305701 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:54:09.305805 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "855ecc3da30b49a6b506d27c78120813"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.305910 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:09.315268 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:09.315882 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:09.329798 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:40957
I20251212 08:54:09.329813 11539 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:40957 every 8 connection(s)
I20251212 08:54:09.331290 11540 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:09.337944 11540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813: Bootstrap starting.
I20251212 08:54:09.339566 11540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:09.347539 11540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813: No bootstrap required, opened a new log
I20251212 08:54:09.348410 11540 raft_consensus.cc:359] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855ecc3da30b49a6b506d27c78120813" member_type: VOTER }
I20251212 08:54:09.348578 11540 raft_consensus.cc:385] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:09.348634 11540 raft_consensus.cc:740] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 855ecc3da30b49a6b506d27c78120813, State: Initialized, Role: FOLLOWER
I20251212 08:54:09.348836 11540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855ecc3da30b49a6b506d27c78120813" member_type: VOTER }
I20251212 08:54:09.348960 11540 raft_consensus.cc:399] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:09.349013 11540 raft_consensus.cc:493] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:09.349080 11540 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:09.359239 11540 raft_consensus.cc:515] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855ecc3da30b49a6b506d27c78120813" member_type: VOTER }
I20251212 08:54:09.359512 11540 leader_election.cc:304] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 855ecc3da30b49a6b506d27c78120813; no voters:
I20251212 08:54:09.359997 11540 leader_election.cc:290] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:09.360142 11543 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:09.360888 11540 sys_catalog.cc:565] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:09.369215 11543 raft_consensus.cc:697] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 1 LEADER]: Becoming Leader. State: Replica: 855ecc3da30b49a6b506d27c78120813, State: Running, Role: LEADER
I20251212 08:54:09.369546 11543 consensus_queue.cc:237] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855ecc3da30b49a6b506d27c78120813" member_type: VOTER }
I20251212 08:54:09.382548 11544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "855ecc3da30b49a6b506d27c78120813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855ecc3da30b49a6b506d27c78120813" member_type: VOTER } }
I20251212 08:54:09.382776 11544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:09.386982 11555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 855ecc3da30b49a6b506d27c78120813. Latest consensus state: current_term: 1 leader_uuid: "855ecc3da30b49a6b506d27c78120813" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "855ecc3da30b49a6b506d27c78120813" member_type: VOTER } }
I20251212 08:54:09.387151 11555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:09.387562 11558 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:09.387732 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:09.389242 11558 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:09.392120 11558 catalog_manager.cc:1357] Generated new cluster ID: 646bdcc8b192463e85910aff30ea1b7f
I20251212 08:54:09.392192 11558 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:09.415532 11558 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:09.416388 11558 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:09.424723 11558 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813: Generated new TSK 0
I20251212 08:54:09.424962 11558 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:09.452754 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:09.455724 11561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.456025 11562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.456570 11564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:09.457435 832 server_base.cc:1047] running on GCE node
I20251212 08:54:09.457701 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:09.457762 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:09.457801 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529649457798 us; error 0 us; skew 500 ppm
I20251212 08:54:09.457960 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:09.459360 832 webserver.cc:492] Webserver started at http://127.0.208.1:39045/ using document root <none> and password file <none>
I20251212 08:54:09.459600 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:09.459679 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:09.459803 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:09.460623 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "319f8909a3a540c2b455f01351e0b49c"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.463837 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20251212 08:54:09.465723 11569 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.466125 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:09.466243 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "319f8909a3a540c2b455f01351e0b49c"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.466361 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ClientTestImmutableColumn.TestUpsert_1.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:09.479693 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:09.480247 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:09.495138 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:09.495250 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.495340 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:09.495379 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.522744 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:38797
I20251212 08:54:09.522760 11639 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:38797 every 8 connection(s)
I20251212 08:54:09.524699 11640 heartbeater.cc:344] Connected to a master server at 127.0.208.62:40957
I20251212 08:54:09.524822 11640 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:09.525144 11640 heartbeater.cc:507] Master 127.0.208.62:40957 requested a full tablet report, sending...
I20251212 08:54:09.525915 11495 ts_manager.cc:194] Registered new tserver with Master: 319f8909a3a540c2b455f01351e0b49c (127.0.208.1:38797)
I20251212 08:54:09.526111 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002269943s
I20251212 08:54:09.527391 11495 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59166
I20251212 08:54:09.537214 11495 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59178:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "imm_val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: true
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:09.539064 11640 heartbeater.cc:499] Master 127.0.208.62:40957 was elected leader, sending a full tablet report...
I20251212 08:54:09.554865 11605 tablet_service.cc:1505] Processing CreateTablet for tablet a661d1f054a34e01b3b5297bb70e63c4 (DEFAULT_TABLE table=client-testtb [id=58d6be1d364545afbfd425a68d514c56]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:09.554967 11604 tablet_service.cc:1505] Processing CreateTablet for tablet d974be739020465282eff86e425dfb54 (DEFAULT_TABLE table=client-testtb [id=58d6be1d364545afbfd425a68d514c56]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:09.555307 11605 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a661d1f054a34e01b3b5297bb70e63c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:09.555555 11604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d974be739020465282eff86e425dfb54. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:09.564803 11650 tablet_bootstrap.cc:492] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c: Bootstrap starting.
I20251212 08:54:09.566107 11650 tablet_bootstrap.cc:654] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:09.567873 11650 tablet_bootstrap.cc:492] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c: No bootstrap required, opened a new log
I20251212 08:54:09.568024 11650 ts_tablet_manager.cc:1403] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:09.568776 11650 raft_consensus.cc:359] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } }
I20251212 08:54:09.568979 11650 raft_consensus.cc:385] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:09.569060 11650 raft_consensus.cc:740] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319f8909a3a540c2b455f01351e0b49c, State: Initialized, Role: FOLLOWER
I20251212 08:54:09.569291 11650 consensus_queue.cc:260] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } }
I20251212 08:54:09.569428 11650 raft_consensus.cc:399] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:09.569486 11650 raft_consensus.cc:493] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:09.569564 11650 raft_consensus.cc:3060] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:09.571813 11650 raft_consensus.cc:515] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } }
I20251212 08:54:09.571992 11650 leader_election.cc:304] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 319f8909a3a540c2b455f01351e0b49c; no voters:
I20251212 08:54:09.572540 11650 leader_election.cc:290] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:09.572633 11652 raft_consensus.cc:2804] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:09.573360 11650 ts_tablet_manager.cc:1434] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20251212 08:54:09.573714 11650 tablet_bootstrap.cc:492] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c: Bootstrap starting.
I20251212 08:54:09.576176 11650 tablet_bootstrap.cc:654] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:09.577495 11652 raft_consensus.cc:697] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 1 LEADER]: Becoming Leader. State: Replica: 319f8909a3a540c2b455f01351e0b49c, State: Running, Role: LEADER
I20251212 08:54:09.577773 11652 consensus_queue.cc:237] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } }
I20251212 08:54:09.578254 11650 tablet_bootstrap.cc:492] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c: No bootstrap required, opened a new log
I20251212 08:54:09.578405 11650 ts_tablet_manager.cc:1403] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251212 08:54:09.579198 11650 raft_consensus.cc:359] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } }
I20251212 08:54:09.579372 11650 raft_consensus.cc:385] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:09.579445 11650 raft_consensus.cc:740] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 319f8909a3a540c2b455f01351e0b49c, State: Initialized, Role: FOLLOWER
I20251212 08:54:09.579706 11650 consensus_queue.cc:260] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } }
I20251212 08:54:09.579900 11650 raft_consensus.cc:399] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:09.579972 11650 raft_consensus.cc:493] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:09.580080 11650 raft_consensus.cc:3060] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:09.580489 11495 catalog_manager.cc:5654] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c reported cstate change: term changed from 0 to 1, leader changed from <none> to 319f8909a3a540c2b455f01351e0b49c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "319f8909a3a540c2b455f01351e0b49c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:09.584043 11650 raft_consensus.cc:515] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } }
I20251212 08:54:09.584302 11650 leader_election.cc:304] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 319f8909a3a540c2b455f01351e0b49c; no voters:
I20251212 08:54:09.584586 11653 raft_consensus.cc:2804] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:09.584704 11653 raft_consensus.cc:697] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 1 LEADER]: Becoming Leader. State: Replica: 319f8909a3a540c2b455f01351e0b49c, State: Running, Role: LEADER
I20251212 08:54:09.584909 11653 consensus_queue.cc:237] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } }
I20251212 08:54:09.585232 11650 leader_election.cc:290] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:09.585731 11650 ts_tablet_manager.cc:1434] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20251212 08:54:09.588028 11495 catalog_manager.cc:5654] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c reported cstate change: term changed from 0 to 1, leader changed from <none> to 319f8909a3a540c2b455f01351e0b49c (127.0.208.1). New cstate: current_term: 1 leader_uuid: "319f8909a3a540c2b455f01351e0b49c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "319f8909a3a540c2b455f01351e0b49c" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38797 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:09.620951 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:09.636659 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:09.637097 832 tablet_replica.cc:333] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c: stopping tablet replica
I20251212 08:54:09.637300 832 raft_consensus.cc:2243] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:09.637509 832 raft_consensus.cc:2272] T d974be739020465282eff86e425dfb54 P 319f8909a3a540c2b455f01351e0b49c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:09.638512 832 tablet_replica.cc:333] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c: stopping tablet replica
I20251212 08:54:09.638659 832 raft_consensus.cc:2243] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:09.638846 832 raft_consensus.cc:2272] T a661d1f054a34e01b3b5297bb70e63c4 P 319f8909a3a540c2b455f01351e0b49c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:09.654220 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:09.671612 832 master.cc:561] Master@127.0.208.62:40957 shutting down...
I20251212 08:54:09.682631 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:09.682902 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:09.683075 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 855ecc3da30b49a6b506d27c78120813: stopping tablet replica
I20251212 08:54:09.687721 832 master.cc:583] Master@127.0.208.62:40957 shutdown complete.
[ OK ] Params/ClientTestImmutableColumn.TestUpsert/1 (409 ms)
[----------] 2 tests from Params/ClientTestImmutableColumn (795 ms total)
[----------] 7 tests from Params/ArrayColumnParamTest
[ RUN ] Params/ArrayColumnParamTest.EmptyAndNullArrays/5
I20251212 08:54:09.701094 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:44843
I20251212 08:54:09.701731 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:09.704250 11662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.704391 11661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.704941 11664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:09.705619 832 server_base.cc:1047] running on GCE node
I20251212 08:54:09.705878 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:09.705928 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:09.705958 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529649705955 us; error 0 us; skew 500 ppm
I20251212 08:54:09.706099 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:09.707298 832 webserver.cc:492] Webserver started at http://127.0.208.62:34667/ using document root <none> and password file <none>
I20251212 08:54:09.707489 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:09.707544 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:09.707664 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:09.708364 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "27eae378871349c4a373e79f0e30332d"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.710805 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:54:09.712250 11669 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.712621 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:09.712710 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "27eae378871349c4a373e79f0e30332d"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.712821 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:09.726769 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:09.727272 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:09.743479 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:44843
I20251212 08:54:09.743492 11730 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:44843 every 8 connection(s)
I20251212 08:54:09.744949 11731 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:09.749670 11731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d: Bootstrap starting.
I20251212 08:54:09.750876 11731 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:09.752440 11731 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d: No bootstrap required, opened a new log
I20251212 08:54:09.753039 11731 raft_consensus.cc:359] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27eae378871349c4a373e79f0e30332d" member_type: VOTER }
I20251212 08:54:09.753158 11731 raft_consensus.cc:385] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:09.753192 11731 raft_consensus.cc:740] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27eae378871349c4a373e79f0e30332d, State: Initialized, Role: FOLLOWER
I20251212 08:54:09.753400 11731 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27eae378871349c4a373e79f0e30332d" member_type: VOTER }
I20251212 08:54:09.753515 11731 raft_consensus.cc:399] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:09.753571 11731 raft_consensus.cc:493] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:09.753646 11731 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:09.755848 11731 raft_consensus.cc:515] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27eae378871349c4a373e79f0e30332d" member_type: VOTER }
I20251212 08:54:09.756029 11731 leader_election.cc:304] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27eae378871349c4a373e79f0e30332d; no voters:
I20251212 08:54:09.756510 11731 leader_election.cc:290] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:09.756625 11734 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:09.757555 11731 sys_catalog.cc:565] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:09.762583 11734 raft_consensus.cc:697] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 1 LEADER]: Becoming Leader. State: Replica: 27eae378871349c4a373e79f0e30332d, State: Running, Role: LEADER
I20251212 08:54:09.762852 11734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27eae378871349c4a373e79f0e30332d" member_type: VOTER }
I20251212 08:54:09.763679 11735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27eae378871349c4a373e79f0e30332d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27eae378871349c4a373e79f0e30332d" member_type: VOTER } }
I20251212 08:54:09.763836 11735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:09.764961 11736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27eae378871349c4a373e79f0e30332d. Latest consensus state: current_term: 1 leader_uuid: "27eae378871349c4a373e79f0e30332d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27eae378871349c4a373e79f0e30332d" member_type: VOTER } }
I20251212 08:54:09.772693 11736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:09.776592 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20251212 08:54:09.778111 11748 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251212 08:54:09.778232 11748 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251212 08:54:09.778718 11749 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:09.779834 11749 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:09.783047 11749 catalog_manager.cc:1357] Generated new cluster ID: 781dd245ce4f4a2c93219264df111c70
I20251212 08:54:09.783140 11749 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:09.832406 11749 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:09.833256 11749 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:09.841464 11749 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d: Generated new TSK 0
I20251212 08:54:09.841722 11749 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:09.843659 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:09.846563 11752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.846582 11753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:09.847218 11755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:09.847576 832 server_base.cc:1047] running on GCE node
I20251212 08:54:09.847792 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:09.847847 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:09.847894 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529649847890 us; error 0 us; skew 500 ppm
I20251212 08:54:09.848042 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:09.849351 832 webserver.cc:492] Webserver started at http://127.0.208.1:33747/ using document root <none> and password file <none>
I20251212 08:54:09.849560 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:09.849627 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:09.849748 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:09.850354 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "e71caf74821f471485142aa4abdcfb1f"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.852788 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:54:09.854308 11760 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.854691 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20251212 08:54:09.854804 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "e71caf74821f471485142aa4abdcfb1f"
format_stamp: "Formatted at 2025-12-12 08:54:09 on dist-test-slave-qdpb"
I20251212 08:54:09.854923 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:09.871404 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:09.871987 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:09.880326 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:09.880429 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.880519 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:09.880569 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:09.917232 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:38121
I20251212 08:54:09.917268 11830 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:38121 every 8 connection(s)
I20251212 08:54:09.919014 11831 heartbeater.cc:344] Connected to a master server at 127.0.208.62:44843
I20251212 08:54:09.919168 11831 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:09.919556 11831 heartbeater.cc:507] Master 127.0.208.62:44843 requested a full tablet report, sending...
I20251212 08:54:09.920389 11686 ts_manager.cc:194] Registered new tserver with Master: e71caf74821f471485142aa4abdcfb1f (127.0.208.1:38121)
I20251212 08:54:09.920558 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.00227317s
I20251212 08:54:09.921905 11686 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59956
I20251212 08:54:09.933415 11831 heartbeater.cc:499] Master 127.0.208.62:44843 was elected leader, sending a full tablet report...
I20251212 08:54:09.935206 11686 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59988:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:09.950747 11791 tablet_service.cc:1505] Processing CreateTablet for tablet 4e026eead59248589b29c58db64b1bb5 (DEFAULT_TABLE table=client-testtb [id=e1606df352714979896ed8946f0e518a]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:09.950747 11790 tablet_service.cc:1505] Processing CreateTablet for tablet aacf8a3d43184a6b97297f305c2c9128 (DEFAULT_TABLE table=client-testtb [id=e1606df352714979896ed8946f0e518a]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:09.951323 11791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e026eead59248589b29c58db64b1bb5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:09.951606 11790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aacf8a3d43184a6b97297f305c2c9128. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:09.958889 11841 tablet_bootstrap.cc:492] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f: Bootstrap starting.
I20251212 08:54:09.960490 11841 tablet_bootstrap.cc:654] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:09.962273 11841 tablet_bootstrap.cc:492] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f: No bootstrap required, opened a new log
I20251212 08:54:09.962399 11841 ts_tablet_manager.cc:1403] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:54:09.963066 11841 raft_consensus.cc:359] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:09.963224 11841 raft_consensus.cc:385] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:09.963275 11841 raft_consensus.cc:740] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e71caf74821f471485142aa4abdcfb1f, State: Initialized, Role: FOLLOWER
I20251212 08:54:09.963467 11841 consensus_queue.cc:260] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:09.963599 11841 raft_consensus.cc:399] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:09.963645 11841 raft_consensus.cc:493] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:09.963711 11841 raft_consensus.cc:3060] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:09.966723 11841 raft_consensus.cc:515] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:09.966943 11841 leader_election.cc:304] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e71caf74821f471485142aa4abdcfb1f; no voters:
I20251212 08:54:09.968523 11841 leader_election.cc:290] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:09.968653 11843 raft_consensus.cc:2804] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:09.972815 11841 ts_tablet_manager.cc:1434] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f: Time spent starting tablet: real 0.010s user 0.007s sys 0.003s
I20251212 08:54:09.973091 11841 tablet_bootstrap.cc:492] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f: Bootstrap starting.
I20251212 08:54:09.973460 11843 raft_consensus.cc:697] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 1 LEADER]: Becoming Leader. State: Replica: e71caf74821f471485142aa4abdcfb1f, State: Running, Role: LEADER
I20251212 08:54:09.973886 11843 consensus_queue.cc:237] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:09.974651 11841 tablet_bootstrap.cc:654] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:09.976517 11686 catalog_manager.cc:5654] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f reported cstate change: term changed from 0 to 1, leader changed from <none> to e71caf74821f471485142aa4abdcfb1f (127.0.208.1). New cstate: current_term: 1 leader_uuid: "e71caf74821f471485142aa4abdcfb1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:09.981927 11841 tablet_bootstrap.cc:492] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f: No bootstrap required, opened a new log
I20251212 08:54:09.982064 11841 ts_tablet_manager.cc:1403] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20251212 08:54:09.982832 11841 raft_consensus.cc:359] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:09.983002 11841 raft_consensus.cc:385] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:09.983065 11841 raft_consensus.cc:740] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e71caf74821f471485142aa4abdcfb1f, State: Initialized, Role: FOLLOWER
I20251212 08:54:09.983269 11841 consensus_queue.cc:260] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:09.983407 11841 raft_consensus.cc:399] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:09.983471 11841 raft_consensus.cc:493] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:09.983542 11841 raft_consensus.cc:3060] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:09.986501 11841 raft_consensus.cc:515] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:09.986714 11841 leader_election.cc:304] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e71caf74821f471485142aa4abdcfb1f; no voters:
I20251212 08:54:09.986876 11841 leader_election.cc:290] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:09.986976 11845 raft_consensus.cc:2804] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:09.987172 11845 raft_consensus.cc:697] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 1 LEADER]: Becoming Leader. State: Replica: e71caf74821f471485142aa4abdcfb1f, State: Running, Role: LEADER
I20251212 08:54:09.987255 11841 ts_tablet_manager.cc:1434] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20251212 08:54:09.987414 11845 consensus_queue.cc:237] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:09.991075 11686 catalog_manager.cc:5654] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f reported cstate change: term changed from 0 to 1, leader changed from <none> to e71caf74821f471485142aa4abdcfb1f (127.0.208.1). New cstate: current_term: 1 leader_uuid: "e71caf74821f471485142aa4abdcfb1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:10.011191 11685 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59988:
name: "table_ea_flush_INT8"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "arr_0"
type: NESTED
is_key: false
is_nullable: true
encoding: PLAIN_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
nested_type {
array {
type: INT8
}
}
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000""\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:10.018494 11791 tablet_service.cc:1505] Processing CreateTablet for tablet 83951908021948c39dfaca2b3726a421 (DEFAULT_TABLE table=table_ea_flush_INT8 [id=52aa695b07f24c208010ca2794f7ee10]), partition=RANGE (key) PARTITION 0 <= VALUES < 100
I20251212 08:54:10.018956 11791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 83951908021948c39dfaca2b3726a421. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:10.024262 11841 tablet_bootstrap.cc:492] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: Bootstrap starting.
I20251212 08:54:10.025810 11841 tablet_bootstrap.cc:654] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:10.027415 11841 tablet_bootstrap.cc:492] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: No bootstrap required, opened a new log
I20251212 08:54:10.027544 11841 ts_tablet_manager.cc:1403] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20251212 08:54:10.028218 11841 raft_consensus.cc:359] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:10.028409 11841 raft_consensus.cc:385] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:10.028468 11841 raft_consensus.cc:740] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e71caf74821f471485142aa4abdcfb1f, State: Initialized, Role: FOLLOWER
I20251212 08:54:10.028653 11841 consensus_queue.cc:260] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:10.028784 11841 raft_consensus.cc:399] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:10.028828 11841 raft_consensus.cc:493] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:10.028896 11841 raft_consensus.cc:3060] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:10.032007 11841 raft_consensus.cc:515] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:10.032223 11841 leader_election.cc:304] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e71caf74821f471485142aa4abdcfb1f; no voters:
I20251212 08:54:10.032425 11841 leader_election.cc:290] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:10.032511 11845 raft_consensus.cc:2804] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:10.032658 11845 raft_consensus.cc:697] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 1 LEADER]: Becoming Leader. State: Replica: e71caf74821f471485142aa4abdcfb1f, State: Running, Role: LEADER
I20251212 08:54:10.032877 11841 ts_tablet_manager.cc:1434] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20251212 08:54:10.032903 11845 consensus_queue.cc:237] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } }
I20251212 08:54:10.038357 11685 catalog_manager.cc:5654] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f reported cstate change: term changed from 0 to 1, leader changed from <none> to e71caf74821f471485142aa4abdcfb1f (127.0.208.1). New cstate: current_term: 1 leader_uuid: "e71caf74821f471485142aa4abdcfb1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e71caf74821f471485142aa4abdcfb1f" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38121 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:10.075525 11685 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59988:
table { table_name: "table_ea_flush_INT8" } modify_external_catalogs: true
I20251212 08:54:10.075733 11685 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59988:
table { table_name: "table_ea_flush_INT8" } modify_external_catalogs: true
I20251212 08:54:10.079071 11685 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d: Sending DeleteTablet for 1 replicas of tablet 83951908021948c39dfaca2b3726a421
I20251212 08:54:10.079705 11791 tablet_service.cc:1552] Processing DeleteTablet for tablet 83951908021948c39dfaca2b3726a421 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:54:10 UTC) from {username='slave'} at 127.0.0.1:59144
I20251212 08:54:10.081511 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:10.083766 11854 tablet_replica.cc:333] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: stopping tablet replica
I20251212 08:54:10.084012 11854 raft_consensus.cc:2243] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:10.084309 11854 raft_consensus.cc:2272] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:10.086293 11854 ts_tablet_manager.cc:1916] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:54:10.097971 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:10.099970 11854 ts_tablet_manager.cc:1929] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251212 08:54:10.100078 11854 log.cc:1199] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_5.1765529557162900-832-0/minicluster-data/ts-0-root/wals/83951908021948c39dfaca2b3726a421
I20251212 08:54:10.100461 11854 ts_tablet_manager.cc:1950] T 83951908021948c39dfaca2b3726a421 P e71caf74821f471485142aa4abdcfb1f: Deleting consensus metadata
W20251212 08:54:10.101277 11673 catalog_manager.cc:4977] TS e71caf74821f471485142aa4abdcfb1f (127.0.208.1:38121): delete failed for tablet 83951908021948c39dfaca2b3726a421 with error code TABLET_NOT_RUNNING: Service unavailable: Tablet Manager is not running: MANAGER_QUIESCING
I20251212 08:54:10.101293 832 tablet_replica.cc:333] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f: stopping tablet replica
I20251212 08:54:10.101557 832 raft_consensus.cc:2243] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:10.101737 832 raft_consensus.cc:2272] T aacf8a3d43184a6b97297f305c2c9128 P e71caf74821f471485142aa4abdcfb1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:10.102769 832 tablet_replica.cc:333] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f: stopping tablet replica
I20251212 08:54:10.102936 832 raft_consensus.cc:2243] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:10.103087 832 raft_consensus.cc:2272] T 4e026eead59248589b29c58db64b1bb5 P e71caf74821f471485142aa4abdcfb1f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:10.118831 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:10.124511 832 master.cc:561] Master@127.0.208.62:44843 shutting down...
W20251212 08:54:10.126143 11673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.0.208.1:38121: connect: Connection refused (error 111) [suppressed 1 similar messages]
W20251212 08:54:10.138283 11673 catalog_manager.cc:4712] TS e71caf74821f471485142aa4abdcfb1f (127.0.208.1:38121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83951908021948c39dfaca2b3726a421: Network error: Client connection negotiation failed: client connection to 127.0.208.1:38121: connect: Connection refused (error 111)
I20251212 08:54:10.221391 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:10.221763 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:10.221989 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 27eae378871349c4a373e79f0e30332d: stopping tablet replica
I20251212 08:54:10.227402 832 master.cc:583] Master@127.0.208.62:44843 shutdown complete.
[ OK ] Params/ArrayColumnParamTest.EmptyAndNullArrays/5 (550 ms)
[ RUN ] Params/ArrayColumnParamTest.EmptyAndNullArrays/13
I20251212 08:54:10.251120 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:32979
I20251212 08:54:10.251679 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:10.254680 11859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:10.254750 11858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:10.255754 11861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:10.256366 832 server_base.cc:1047] running on GCE node
I20251212 08:54:10.256680 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:10.256748 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:10.256794 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529650256791 us; error 0 us; skew 500 ppm
I20251212 08:54:10.256960 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:10.258314 832 webserver.cc:492] Webserver started at http://127.0.208.62:43841/ using document root <none> and password file <none>
I20251212 08:54:10.258551 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:10.258626 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:10.258752 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:10.259558 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "1811d92f76dc4f11afc5ee719ec2f390"
format_stamp: "Formatted at 2025-12-12 08:54:10 on dist-test-slave-qdpb"
I20251212 08:54:10.262995 832 fs_manager.cc:696] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20251212 08:54:10.264942 11866 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:10.265365 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:54:10.265475 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "1811d92f76dc4f11afc5ee719ec2f390"
format_stamp: "Formatted at 2025-12-12 08:54:10 on dist-test-slave-qdpb"
I20251212 08:54:10.265599 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:10.282833 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:10.283413 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:10.298013 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:32979
I20251212 08:54:10.298027 11927 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:32979 every 8 connection(s)
I20251212 08:54:10.299736 11928 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:10.305050 11928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390: Bootstrap starting.
I20251212 08:54:10.306376 11928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:10.307994 11928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390: No bootstrap required, opened a new log
I20251212 08:54:10.308658 11928 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1811d92f76dc4f11afc5ee719ec2f390" member_type: VOTER }
I20251212 08:54:10.308789 11928 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:10.308842 11928 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1811d92f76dc4f11afc5ee719ec2f390, State: Initialized, Role: FOLLOWER
I20251212 08:54:10.309036 11928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1811d92f76dc4f11afc5ee719ec2f390" member_type: VOTER }
I20251212 08:54:10.309144 11928 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:10.309206 11928 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:10.309280 11928 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:10.311337 11928 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1811d92f76dc4f11afc5ee719ec2f390" member_type: VOTER }
I20251212 08:54:10.311504 11928 leader_election.cc:304] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1811d92f76dc4f11afc5ee719ec2f390; no voters:
I20251212 08:54:10.311950 11928 leader_election.cc:290] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:10.312064 11931 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:10.313102 11928 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:10.316471 11931 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 1 LEADER]: Becoming Leader. State: Replica: 1811d92f76dc4f11afc5ee719ec2f390, State: Running, Role: LEADER
I20251212 08:54:10.316793 11931 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1811d92f76dc4f11afc5ee719ec2f390" member_type: VOTER }
I20251212 08:54:10.317799 11932 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1811d92f76dc4f11afc5ee719ec2f390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1811d92f76dc4f11afc5ee719ec2f390" member_type: VOTER } }
I20251212 08:54:10.317982 11932 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:10.317834 11933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1811d92f76dc4f11afc5ee719ec2f390. Latest consensus state: current_term: 1 leader_uuid: "1811d92f76dc4f11afc5ee719ec2f390" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1811d92f76dc4f11afc5ee719ec2f390" member_type: VOTER } }
I20251212 08:54:10.318181 11933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:10.326295 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20251212 08:54:10.326995 11947 catalog_manager.cc:1568] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20251212 08:54:10.327075 11947 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20251212 08:54:10.327163 11941 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:10.328069 11941 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:10.331125 11941 catalog_manager.cc:1357] Generated new cluster ID: 2586745ac6754b8599e4659b87575617
I20251212 08:54:10.331207 11941 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:10.342953 11941 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:10.343828 11941 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:10.353240 11941 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390: Generated new TSK 0
I20251212 08:54:10.353516 11941 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:10.359308 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:10.362095 11949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:10.362325 11950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:10.363174 11952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:10.363330 832 server_base.cc:1047] running on GCE node
I20251212 08:54:10.363788 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:10.363848 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:10.363873 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529650363869 us; error 0 us; skew 500 ppm
I20251212 08:54:10.364013 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:10.369432 832 webserver.cc:492] Webserver started at http://127.0.208.1:42519/ using document root <none> and password file <none>
I20251212 08:54:10.369660 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:10.369724 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:10.369848 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:10.370513 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "bf6d60fcc8d84e6a9e6c3344f722735b"
format_stamp: "Formatted at 2025-12-12 08:54:10 on dist-test-slave-qdpb"
I20251212 08:54:10.373155 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20251212 08:54:10.374753 11957 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:10.375102 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:54:10.375195 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "bf6d60fcc8d84e6a9e6c3344f722735b"
format_stamp: "Formatted at 2025-12-12 08:54:10 on dist-test-slave-qdpb"
I20251212 08:54:10.375308 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:10.395069 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:10.395699 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:10.397425 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:10.397505 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:10.397584 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:10.397631 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:10.432572 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:36395
I20251212 08:54:10.432586 12027 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:36395 every 8 connection(s)
I20251212 08:54:10.434751 12028 heartbeater.cc:344] Connected to a master server at 127.0.208.62:32979
I20251212 08:54:10.434859 12028 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:10.435217 12028 heartbeater.cc:507] Master 127.0.208.62:32979 requested a full tablet report, sending...
I20251212 08:54:10.436127 11883 ts_manager.cc:194] Registered new tserver with Master: bf6d60fcc8d84e6a9e6c3344f722735b (127.0.208.1:36395)
I20251212 08:54:10.436930 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002408123s
I20251212 08:54:10.437644 11883 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:40010
I20251212 08:54:10.449242 12028 heartbeater.cc:499] Master 127.0.208.62:32979 was elected leader, sending a full tablet report...
I20251212 08:54:10.449565 11883 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40028:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:10.467464 11987 tablet_service.cc:1505] Processing CreateTablet for tablet a9fc094f942a4eeeb942b1d517336532 (DEFAULT_TABLE table=client-testtb [id=5b59169d535d495cbdefaa4863a62a82]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:10.467482 11988 tablet_service.cc:1505] Processing CreateTablet for tablet 951d4d9385dd4ab5a7fdd45cbd7c8308 (DEFAULT_TABLE table=client-testtb [id=5b59169d535d495cbdefaa4863a62a82]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:10.468034 11987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9fc094f942a4eeeb942b1d517336532. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:10.468274 11988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 951d4d9385dd4ab5a7fdd45cbd7c8308. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:10.480351 12038 tablet_bootstrap.cc:492] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b: Bootstrap starting.
I20251212 08:54:10.482685 12038 tablet_bootstrap.cc:654] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:10.494014 12038 tablet_bootstrap.cc:492] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b: No bootstrap required, opened a new log
I20251212 08:54:10.494267 12038 ts_tablet_manager.cc:1403] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.009s
I20251212 08:54:10.495227 12038 raft_consensus.cc:359] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.495425 12038 raft_consensus.cc:385] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:10.495486 12038 raft_consensus.cc:740] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6d60fcc8d84e6a9e6c3344f722735b, State: Initialized, Role: FOLLOWER
I20251212 08:54:10.495731 12038 consensus_queue.cc:260] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.495906 12038 raft_consensus.cc:399] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:10.495975 12038 raft_consensus.cc:493] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:10.496060 12038 raft_consensus.cc:3060] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:10.499817 12038 raft_consensus.cc:515] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.500099 12038 leader_election.cc:304] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf6d60fcc8d84e6a9e6c3344f722735b; no voters:
I20251212 08:54:10.500623 12038 leader_election.cc:290] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:10.500766 12040 raft_consensus.cc:2804] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:10.501199 12040 raft_consensus.cc:697] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 LEADER]: Becoming Leader. State: Replica: bf6d60fcc8d84e6a9e6c3344f722735b, State: Running, Role: LEADER
I20251212 08:54:10.501490 12040 consensus_queue.cc:237] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.504237 11882 catalog_manager.cc:5654] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b reported cstate change: term changed from 0 to 1, leader changed from <none> to bf6d60fcc8d84e6a9e6c3344f722735b (127.0.208.1). New cstate: current_term: 1 leader_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:10.512898 12038 ts_tablet_manager.cc:1434] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b: Time spent starting tablet: real 0.019s user 0.007s sys 0.012s
I20251212 08:54:10.513288 12038 tablet_bootstrap.cc:492] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b: Bootstrap starting.
I20251212 08:54:10.515563 12038 tablet_bootstrap.cc:654] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:10.518296 12038 tablet_bootstrap.cc:492] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b: No bootstrap required, opened a new log
I20251212 08:54:10.518477 12038 ts_tablet_manager.cc:1403] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20251212 08:54:10.519626 12038 raft_consensus.cc:359] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.519850 12038 raft_consensus.cc:385] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:10.519917 12038 raft_consensus.cc:740] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6d60fcc8d84e6a9e6c3344f722735b, State: Initialized, Role: FOLLOWER
I20251212 08:54:10.520180 12038 consensus_queue.cc:260] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.520419 12038 raft_consensus.cc:399] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:10.520479 12038 raft_consensus.cc:493] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:10.537832 12038 raft_consensus.cc:3060] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:10.541356 12038 raft_consensus.cc:515] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.541654 12038 leader_election.cc:304] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf6d60fcc8d84e6a9e6c3344f722735b; no voters:
I20251212 08:54:10.541867 12038 leader_election.cc:290] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:10.541937 12040 raft_consensus.cc:2804] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:10.542383 12038 ts_tablet_manager.cc:1434] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b: Time spent starting tablet: real 0.024s user 0.022s sys 0.001s
I20251212 08:54:10.542595 12040 raft_consensus.cc:697] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 LEADER]: Becoming Leader. State: Replica: bf6d60fcc8d84e6a9e6c3344f722735b, State: Running, Role: LEADER
I20251212 08:54:10.542858 12040 consensus_queue.cc:237] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.545477 11882 catalog_manager.cc:5654] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b reported cstate change: term changed from 0 to 1, leader changed from <none> to bf6d60fcc8d84e6a9e6c3344f722735b (127.0.208.1). New cstate: current_term: 1 leader_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:10.568152 11882 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40028:
name: "table_ea_flush_INT32"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "arr_0"
type: NESTED
is_key: false
is_nullable: true
encoding: PLAIN_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
nested_type {
array {
type: INT32
}
}
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000""\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:10.576429 11988 tablet_service.cc:1505] Processing CreateTablet for tablet dd8a3da3a8844f7890d266ac1f75b0e5 (DEFAULT_TABLE table=table_ea_flush_INT32 [id=7942a2532ce8446c9d5d25cd1d7436a2]), partition=RANGE (key) PARTITION 0 <= VALUES < 100
I20251212 08:54:10.576903 11988 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd8a3da3a8844f7890d266ac1f75b0e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:10.583823 12038 tablet_bootstrap.cc:492] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: Bootstrap starting.
I20251212 08:54:10.585603 12038 tablet_bootstrap.cc:654] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:10.587337 12038 tablet_bootstrap.cc:492] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: No bootstrap required, opened a new log
I20251212 08:54:10.587489 12038 ts_tablet_manager.cc:1403] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:54:10.588063 12038 raft_consensus.cc:359] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.588202 12038 raft_consensus.cc:385] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:10.588269 12038 raft_consensus.cc:740] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf6d60fcc8d84e6a9e6c3344f722735b, State: Initialized, Role: FOLLOWER
I20251212 08:54:10.588531 12038 consensus_queue.cc:260] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.588653 12038 raft_consensus.cc:399] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:10.588719 12038 raft_consensus.cc:493] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:10.588794 12038 raft_consensus.cc:3060] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:10.591114 12038 raft_consensus.cc:515] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.591295 12038 leader_election.cc:304] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf6d60fcc8d84e6a9e6c3344f722735b; no voters:
I20251212 08:54:10.591467 12038 leader_election.cc:290] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:10.591553 12041 raft_consensus.cc:2804] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:10.591696 12041 raft_consensus.cc:697] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 LEADER]: Becoming Leader. State: Replica: bf6d60fcc8d84e6a9e6c3344f722735b, State: Running, Role: LEADER
I20251212 08:54:10.591925 12041 consensus_queue.cc:237] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } }
I20251212 08:54:10.591913 12038 ts_tablet_manager.cc:1434] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20251212 08:54:10.594359 11883 catalog_manager.cc:5654] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b reported cstate change: term changed from 0 to 1, leader changed from <none> to bf6d60fcc8d84e6a9e6c3344f722735b (127.0.208.1). New cstate: current_term: 1 leader_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf6d60fcc8d84e6a9e6c3344f722735b" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 36395 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:10.631184 11883 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40028:
table { table_name: "table_ea_flush_INT32" } modify_external_catalogs: true
I20251212 08:54:10.631376 11883 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40028:
table { table_name: "table_ea_flush_INT32" } modify_external_catalogs: true
I20251212 08:54:10.634313 11883 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390: Sending DeleteTablet for 1 replicas of tablet dd8a3da3a8844f7890d266ac1f75b0e5
I20251212 08:54:10.635008 11988 tablet_service.cc:1552] Processing DeleteTablet for tablet dd8a3da3a8844f7890d266ac1f75b0e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:54:10 UTC) from {username='slave'} at 127.0.0.1:55264
I20251212 08:54:10.635843 12051 tablet_replica.cc:333] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: stopping tablet replica
I20251212 08:54:10.636232 12051 raft_consensus.cc:2243] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:10.636660 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:10.640152 12051 raft_consensus.cc:2272] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:10.641505 12051 ts_tablet_manager.cc:1916] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:54:10.652242 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:10.653815 12051 ts_tablet_manager.cc:1929] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251212 08:54:10.653926 12051 log.cc:1199] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_13.1765529557162900-832-0/minicluster-data/ts-0-root/wals/dd8a3da3a8844f7890d266ac1f75b0e5
I20251212 08:54:10.654223 12051 ts_tablet_manager.cc:1950] T dd8a3da3a8844f7890d266ac1f75b0e5 P bf6d60fcc8d84e6a9e6c3344f722735b: Deleting consensus metadata
I20251212 08:54:10.655037 832 tablet_replica.cc:333] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b: stopping tablet replica
I20251212 08:54:10.655244 832 raft_consensus.cc:2243] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:10.655426 832 raft_consensus.cc:2272] T a9fc094f942a4eeeb942b1d517336532 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:10.656247 832 tablet_replica.cc:333] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b: stopping tablet replica
I20251212 08:54:10.656421 832 raft_consensus.cc:2243] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:10.656561 832 raft_consensus.cc:2272] T 951d4d9385dd4ab5a7fdd45cbd7c8308 P bf6d60fcc8d84e6a9e6c3344f722735b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:10.661401 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:10.666761 832 master.cc:561] Master@127.0.208.62:32979 shutting down...
I20251212 08:54:10.690495 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:10.690739 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:10.690927 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1811d92f76dc4f11afc5ee719ec2f390: stopping tablet replica
I20251212 08:54:10.705500 832 master.cc:583] Master@127.0.208.62:32979 shutdown complete.
[ OK ] Params/ArrayColumnParamTest.EmptyAndNullArrays/13 (465 ms)
[ RUN ] Params/ArrayColumnParamTest.EmptyAndNullArrays/21
I20251212 08:54:10.716219 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:36501
I20251212 08:54:10.716802 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:10.719269 12056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:10.719327 12055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:10.719997 12058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:10.720018 832 server_base.cc:1047] running on GCE node
I20251212 08:54:10.720418 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:10.720479 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:10.720522 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529650720518 us; error 0 us; skew 500 ppm
I20251212 08:54:10.720672 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:10.721828 832 webserver.cc:492] Webserver started at http://127.0.208.62:41797/ using document root <none> and password file <none>
I20251212 08:54:10.722034 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:10.722182 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:10.722352 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:10.722988 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "fb4890e229704a9b8e1c7f39684a77ca"
format_stamp: "Formatted at 2025-12-12 08:54:10 on dist-test-slave-qdpb"
I20251212 08:54:10.725657 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:10.727312 12063 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:10.727694 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:54:10.727795 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "fb4890e229704a9b8e1c7f39684a77ca"
format_stamp: "Formatted at 2025-12-12 08:54:10 on dist-test-slave-qdpb"
I20251212 08:54:10.727918 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:10.740088 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:10.740684 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:10.752348 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:36501
I20251212 08:54:10.752364 12124 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:36501 every 8 connection(s)
I20251212 08:54:10.753621 12125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:10.758462 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca: Bootstrap starting.
I20251212 08:54:10.759701 12125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:10.761264 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca: No bootstrap required, opened a new log
I20251212 08:54:10.761883 12125 raft_consensus.cc:359] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4890e229704a9b8e1c7f39684a77ca" member_type: VOTER }
I20251212 08:54:10.762005 12125 raft_consensus.cc:385] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:10.762061 12125 raft_consensus.cc:740] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4890e229704a9b8e1c7f39684a77ca, State: Initialized, Role: FOLLOWER
I20251212 08:54:10.762269 12125 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4890e229704a9b8e1c7f39684a77ca" member_type: VOTER }
I20251212 08:54:10.762370 12125 raft_consensus.cc:399] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:10.762427 12125 raft_consensus.cc:493] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:10.762502 12125 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:10.764626 12125 raft_consensus.cc:515] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4890e229704a9b8e1c7f39684a77ca" member_type: VOTER }
I20251212 08:54:10.764809 12125 leader_election.cc:304] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb4890e229704a9b8e1c7f39684a77ca; no voters:
I20251212 08:54:10.765223 12125 leader_election.cc:290] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:10.765385 12128 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:10.765821 12128 raft_consensus.cc:697] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 1 LEADER]: Becoming Leader. State: Replica: fb4890e229704a9b8e1c7f39684a77ca, State: Running, Role: LEADER
I20251212 08:54:10.766057 12125 sys_catalog.cc:565] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:10.766048 12128 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4890e229704a9b8e1c7f39684a77ca" member_type: VOTER }
I20251212 08:54:10.766896 12129 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb4890e229704a9b8e1c7f39684a77ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4890e229704a9b8e1c7f39684a77ca" member_type: VOTER } }
I20251212 08:54:10.766948 12130 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb4890e229704a9b8e1c7f39684a77ca. Latest consensus state: current_term: 1 leader_uuid: "fb4890e229704a9b8e1c7f39684a77ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb4890e229704a9b8e1c7f39684a77ca" member_type: VOTER } }
I20251212 08:54:10.767185 12130 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:10.767278 12129 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:10.768051 12133 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:10.769142 12133 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:10.771446 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:10.772198 12133 catalog_manager.cc:1357] Generated new cluster ID: ed6520c223f34768b2fe57af02ed8384
I20251212 08:54:10.772305 12133 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:10.788277 12133 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:10.789206 12133 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:10.803470 12133 catalog_manager.cc:6027] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca: Generated new TSK 0
I20251212 08:54:10.803771 12133 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:10.836630 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:10.839686 12146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:10.839921 12149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:10.840011 832 server_base.cc:1047] running on GCE node
W20251212 08:54:10.840011 12147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:10.840593 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:10.840660 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:10.840699 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529650840695 us; error 0 us; skew 500 ppm
I20251212 08:54:10.840842 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:10.842052 832 webserver.cc:492] Webserver started at http://127.0.208.1:37267/ using document root <none> and password file <none>
I20251212 08:54:10.842263 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:10.842352 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:10.842478 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:10.843094 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "2b9efdf9e3cb467789841a9d69714188"
format_stamp: "Formatted at 2025-12-12 08:54:10 on dist-test-slave-qdpb"
I20251212 08:54:10.845691 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:10.847364 12154 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:10.847752 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:54:10.847880 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "2b9efdf9e3cb467789841a9d69714188"
format_stamp: "Formatted at 2025-12-12 08:54:10 on dist-test-slave-qdpb"
I20251212 08:54:10.847986 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:10.878540 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:10.879083 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:10.881183 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:10.881269 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:10.881374 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:10.881429 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:10.903057 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:37017
I20251212 08:54:10.903059 12224 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:37017 every 8 connection(s)
I20251212 08:54:10.904758 12225 heartbeater.cc:344] Connected to a master server at 127.0.208.62:36501
I20251212 08:54:10.904891 12225 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:10.905186 12225 heartbeater.cc:507] Master 127.0.208.62:36501 requested a full tablet report, sending...
I20251212 08:54:10.905932 12080 ts_manager.cc:194] Registered new tserver with Master: 2b9efdf9e3cb467789841a9d69714188 (127.0.208.1:37017)
I20251212 08:54:10.906379 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002288049s
I20251212 08:54:10.906949 12080 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:51570
I20251212 08:54:10.917122 12080 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51600:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:10.918159 12225 heartbeater.cc:499] Master 127.0.208.62:36501 was elected leader, sending a full tablet report...
I20251212 08:54:10.930014 12185 tablet_service.cc:1505] Processing CreateTablet for tablet f0c5f7b4acc4407896c31a95f1e3e423 (DEFAULT_TABLE table=client-testtb [id=5cedc71bc9c044ab9d00e64310e32362]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:10.930059 12184 tablet_service.cc:1505] Processing CreateTablet for tablet 97d931003a4f4aada7c0099ca9de885e (DEFAULT_TABLE table=client-testtb [id=5cedc71bc9c044ab9d00e64310e32362]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:10.930608 12185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0c5f7b4acc4407896c31a95f1e3e423. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:10.930802 12184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 97d931003a4f4aada7c0099ca9de885e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:10.938374 12235 tablet_bootstrap.cc:492] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188: Bootstrap starting.
I20251212 08:54:10.939761 12235 tablet_bootstrap.cc:654] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:10.941388 12235 tablet_bootstrap.cc:492] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188: No bootstrap required, opened a new log
I20251212 08:54:10.941557 12235 ts_tablet_manager.cc:1403] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:10.942348 12235 raft_consensus.cc:359] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.942526 12235 raft_consensus.cc:385] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:10.942592 12235 raft_consensus.cc:740] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9efdf9e3cb467789841a9d69714188, State: Initialized, Role: FOLLOWER
I20251212 08:54:10.942816 12235 consensus_queue.cc:260] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.942936 12235 raft_consensus.cc:399] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:10.943001 12235 raft_consensus.cc:493] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:10.943084 12235 raft_consensus.cc:3060] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:10.945570 12235 raft_consensus.cc:515] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.945765 12235 leader_election.cc:304] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b9efdf9e3cb467789841a9d69714188; no voters:
I20251212 08:54:10.946143 12235 leader_election.cc:290] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:10.946288 12237 raft_consensus.cc:2804] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:10.946723 12237 raft_consensus.cc:697] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 1 LEADER]: Becoming Leader. State: Replica: 2b9efdf9e3cb467789841a9d69714188, State: Running, Role: LEADER
I20251212 08:54:10.946792 12235 ts_tablet_manager.cc:1434] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20251212 08:54:10.946960 12237 consensus_queue.cc:237] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.947062 12235 tablet_bootstrap.cc:492] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188: Bootstrap starting.
I20251212 08:54:10.948637 12235 tablet_bootstrap.cc:654] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:10.949051 12079 catalog_manager.cc:5654] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b9efdf9e3cb467789841a9d69714188 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "2b9efdf9e3cb467789841a9d69714188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:10.950433 12235 tablet_bootstrap.cc:492] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188: No bootstrap required, opened a new log
I20251212 08:54:10.950552 12235 ts_tablet_manager.cc:1403] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:54:10.951164 12235 raft_consensus.cc:359] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.951279 12235 raft_consensus.cc:385] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:10.951314 12235 raft_consensus.cc:740] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9efdf9e3cb467789841a9d69714188, State: Initialized, Role: FOLLOWER
I20251212 08:54:10.951514 12235 consensus_queue.cc:260] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.951640 12235 raft_consensus.cc:399] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:10.951683 12235 raft_consensus.cc:493] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:10.951737 12235 raft_consensus.cc:3060] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:10.954069 12235 raft_consensus.cc:515] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.954273 12235 leader_election.cc:304] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b9efdf9e3cb467789841a9d69714188; no voters:
I20251212 08:54:10.954456 12235 leader_election.cc:290] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:10.954536 12237 raft_consensus.cc:2804] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:10.954694 12237 raft_consensus.cc:697] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 1 LEADER]: Becoming Leader. State: Replica: 2b9efdf9e3cb467789841a9d69714188, State: Running, Role: LEADER
I20251212 08:54:10.954818 12235 ts_tablet_manager.cc:1434] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:54:10.954949 12237 consensus_queue.cc:237] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.957188 12079 catalog_manager.cc:5654] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b9efdf9e3cb467789841a9d69714188 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "2b9efdf9e3cb467789841a9d69714188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:10.968210 12079 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51600:
name: "table_ea_flush_BINARY"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "arr_0"
type: NESTED
is_key: false
is_nullable: true
encoding: PLAIN_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
nested_type {
array {
type: BINARY
}
}
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000""\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:10.975730 12185 tablet_service.cc:1505] Processing CreateTablet for tablet 900410df190142729cb23415c4350a23 (DEFAULT_TABLE table=table_ea_flush_BINARY [id=7b775313e50f4fbc93c7ce8d2ac51228]), partition=RANGE (key) PARTITION 0 <= VALUES < 100
I20251212 08:54:10.976112 12185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 900410df190142729cb23415c4350a23. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:10.981259 12235 tablet_bootstrap.cc:492] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: Bootstrap starting.
I20251212 08:54:10.982544 12235 tablet_bootstrap.cc:654] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:10.984038 12235 tablet_bootstrap.cc:492] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: No bootstrap required, opened a new log
I20251212 08:54:10.984200 12235 ts_tablet_manager.cc:1403] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:10.985245 12235 raft_consensus.cc:359] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.985430 12235 raft_consensus.cc:385] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:10.985510 12235 raft_consensus.cc:740] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b9efdf9e3cb467789841a9d69714188, State: Initialized, Role: FOLLOWER
I20251212 08:54:10.985711 12235 consensus_queue.cc:260] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.985855 12235 raft_consensus.cc:399] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:10.985900 12235 raft_consensus.cc:493] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:10.985953 12235 raft_consensus.cc:3060] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:10.987994 12235 raft_consensus.cc:515] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.988178 12235 leader_election.cc:304] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b9efdf9e3cb467789841a9d69714188; no voters:
I20251212 08:54:10.988384 12235 leader_election.cc:290] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:10.988474 12237 raft_consensus.cc:2804] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:10.988646 12237 raft_consensus.cc:697] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 1 LEADER]: Becoming Leader. State: Replica: 2b9efdf9e3cb467789841a9d69714188, State: Running, Role: LEADER
I20251212 08:54:10.988879 12237 consensus_queue.cc:237] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } }
I20251212 08:54:10.989193 12235 ts_tablet_manager.cc:1434] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20251212 08:54:10.990921 12079 catalog_manager.cc:5654] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b9efdf9e3cb467789841a9d69714188 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "2b9efdf9e3cb467789841a9d69714188" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b9efdf9e3cb467789841a9d69714188" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37017 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:11.032255 12079 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51600:
table { table_name: "table_ea_flush_BINARY" } modify_external_catalogs: true
I20251212 08:54:11.032496 12079 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51600:
table { table_name: "table_ea_flush_BINARY" } modify_external_catalogs: true
I20251212 08:54:11.035442 12079 catalog_manager.cc:5941] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca: Sending DeleteTablet for 1 replicas of tablet 900410df190142729cb23415c4350a23
I20251212 08:54:11.036164 12185 tablet_service.cc:1552] Processing DeleteTablet for tablet 900410df190142729cb23415c4350a23 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:54:11 UTC) from {username='slave'} at 127.0.0.1:59164
I20251212 08:54:11.036929 12248 tablet_replica.cc:333] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: stopping tablet replica
I20251212 08:54:11.037225 12248 raft_consensus.cc:2243] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.037464 12248 raft_consensus.cc:2272] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.038017 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:11.038501 12248 ts_tablet_manager.cc:1916] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:54:11.050719 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:11.051601 12248 ts_tablet_manager.cc:1929] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251212 08:54:11.051697 12248 log.cc:1199] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_21.1765529557162900-832-0/minicluster-data/ts-0-root/wals/900410df190142729cb23415c4350a23
I20251212 08:54:11.051975 12248 ts_tablet_manager.cc:1950] T 900410df190142729cb23415c4350a23 P 2b9efdf9e3cb467789841a9d69714188: Deleting consensus metadata
I20251212 08:54:11.052688 832 tablet_replica.cc:333] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188: stopping tablet replica
I20251212 08:54:11.052868 832 raft_consensus.cc:2243] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.053001 832 raft_consensus.cc:2272] T f0c5f7b4acc4407896c31a95f1e3e423 P 2b9efdf9e3cb467789841a9d69714188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.053779 832 tablet_replica.cc:333] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188: stopping tablet replica
I20251212 08:54:11.053885 832 raft_consensus.cc:2243] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.053992 832 raft_consensus.cc:2272] T 97d931003a4f4aada7c0099ca9de885e P 2b9efdf9e3cb467789841a9d69714188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.069597 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:11.085372 832 master.cc:561] Master@127.0.208.62:36501 shutting down...
I20251212 08:54:11.112475 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.112803 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.113067 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P fb4890e229704a9b8e1c7f39684a77ca: stopping tablet replica
I20251212 08:54:11.127525 832 master.cc:583] Master@127.0.208.62:36501 shutdown complete.
[ OK ] Params/ArrayColumnParamTest.EmptyAndNullArrays/21 (421 ms)
[ RUN ] Params/ArrayColumnParamTest.EmptyAndNullArrays/29
I20251212 08:54:11.138346 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:33025
I20251212 08:54:11.138860 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:11.141258 12252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.141342 12253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.141772 12255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:11.142177 832 server_base.cc:1047] running on GCE node
I20251212 08:54:11.142406 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:11.142457 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:11.142478 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529651142475 us; error 0 us; skew 500 ppm
I20251212 08:54:11.142611 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:11.143780 832 webserver.cc:492] Webserver started at http://127.0.208.62:45471/ using document root <none> and password file <none>
I20251212 08:54:11.143980 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:11.144054 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:11.144177 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:11.144821 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "daa51d87af584c01bed44664fd1d304d"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.147253 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:11.148748 12260 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:11.149119 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:11.149204 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "daa51d87af584c01bed44664fd1d304d"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.149310 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:11.169306 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:11.169816 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:11.183590 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:33025
I20251212 08:54:11.183622 12321 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:33025 every 8 connection(s)
I20251212 08:54:11.184916 12322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:11.189826 12322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d: Bootstrap starting.
I20251212 08:54:11.190977 12322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:11.192447 12322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d: No bootstrap required, opened a new log
I20251212 08:54:11.193012 12322 raft_consensus.cc:359] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa51d87af584c01bed44664fd1d304d" member_type: VOTER }
I20251212 08:54:11.193152 12322 raft_consensus.cc:385] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:11.193188 12322 raft_consensus.cc:740] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daa51d87af584c01bed44664fd1d304d, State: Initialized, Role: FOLLOWER
I20251212 08:54:11.193364 12322 consensus_queue.cc:260] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa51d87af584c01bed44664fd1d304d" member_type: VOTER }
I20251212 08:54:11.193481 12322 raft_consensus.cc:399] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:11.193518 12322 raft_consensus.cc:493] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:11.193572 12322 raft_consensus.cc:3060] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:11.195555 12322 raft_consensus.cc:515] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa51d87af584c01bed44664fd1d304d" member_type: VOTER }
I20251212 08:54:11.195711 12322 leader_election.cc:304] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: daa51d87af584c01bed44664fd1d304d; no voters:
I20251212 08:54:11.196030 12322 leader_election.cc:290] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:11.196158 12325 raft_consensus.cc:2804] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:11.196599 12325 raft_consensus.cc:697] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 1 LEADER]: Becoming Leader. State: Replica: daa51d87af584c01bed44664fd1d304d, State: Running, Role: LEADER
I20251212 08:54:11.196820 12325 consensus_queue.cc:237] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa51d87af584c01bed44664fd1d304d" member_type: VOTER }
I20251212 08:54:11.196897 12322 sys_catalog.cc:565] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:11.197634 12326 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "daa51d87af584c01bed44664fd1d304d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa51d87af584c01bed44664fd1d304d" member_type: VOTER } }
I20251212 08:54:11.197731 12327 sys_catalog.cc:455] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [sys.catalog]: SysCatalogTable state changed. Reason: New leader daa51d87af584c01bed44664fd1d304d. Latest consensus state: current_term: 1 leader_uuid: "daa51d87af584c01bed44664fd1d304d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "daa51d87af584c01bed44664fd1d304d" member_type: VOTER } }
I20251212 08:54:11.197903 12327 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:11.197953 12326 sys_catalog.cc:458] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:11.199039 12330 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:11.200932 12330 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:11.203735 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:11.203856 12330 catalog_manager.cc:1357] Generated new cluster ID: 0174c489524343dd8b1a3dcc600e4333
I20251212 08:54:11.203936 12330 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:11.211738 12330 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:11.212499 12330 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:11.225576 12330 catalog_manager.cc:6027] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d: Generated new TSK 0
I20251212 08:54:11.225826 12330 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:11.236672 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:11.239991 12343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.239989 12344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.240401 12346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:11.240628 832 server_base.cc:1047] running on GCE node
I20251212 08:54:11.240852 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:11.240901 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:11.240923 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529651240920 us; error 0 us; skew 500 ppm
I20251212 08:54:11.241065 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:11.242239 832 webserver.cc:492] Webserver started at http://127.0.208.1:40523/ using document root <none> and password file <none>
I20251212 08:54:11.242419 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:11.242476 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:11.242563 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:11.243149 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.245605 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.002s
I20251212 08:54:11.247135 12351 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:11.247551 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:11.247644 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.247740 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:11.265527 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:11.265995 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:11.267472 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:11.267550 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:11.267650 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:11.267721 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:11.284762 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:43253
I20251212 08:54:11.284781 12421 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:43253 every 8 connection(s)
I20251212 08:54:11.286206 12422 heartbeater.cc:344] Connected to a master server at 127.0.208.62:33025
I20251212 08:54:11.286327 12422 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:11.286620 12422 heartbeater.cc:507] Master 127.0.208.62:33025 requested a full tablet report, sending...
I20251212 08:54:11.287385 12277 ts_manager.cc:194] Registered new tserver with Master: 4dd7e4429e0d4b5faad3e02e1e12f38d (127.0.208.1:43253)
I20251212 08:54:11.287902 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002241274s
I20251212 08:54:11.288426 12277 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47082
I20251212 08:54:11.297574 12277 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47090:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:11.299832 12422 heartbeater.cc:499] Master 127.0.208.62:33025 was elected leader, sending a full tablet report...
I20251212 08:54:11.310271 12382 tablet_service.cc:1505] Processing CreateTablet for tablet 5e2f4f86ed7a49a7bea53f37df2f0927 (DEFAULT_TABLE table=client-testtb [id=fdff20a271d44dcea6f774444b71efaa]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:11.310371 12381 tablet_service.cc:1505] Processing CreateTablet for tablet e1203ea820394161a9805e4d38a143ef (DEFAULT_TABLE table=client-testtb [id=fdff20a271d44dcea6f774444b71efaa]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:11.310843 12382 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e2f4f86ed7a49a7bea53f37df2f0927. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:11.311020 12381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1203ea820394161a9805e4d38a143ef. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:11.318907 12432 tablet_bootstrap.cc:492] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d: Bootstrap starting.
I20251212 08:54:11.320168 12432 tablet_bootstrap.cc:654] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:11.321655 12432 tablet_bootstrap.cc:492] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d: No bootstrap required, opened a new log
I20251212 08:54:11.321786 12432 ts_tablet_manager.cc:1403] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:11.322367 12432 raft_consensus.cc:359] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.322506 12432 raft_consensus.cc:385] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:11.322566 12432 raft_consensus.cc:740] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd7e4429e0d4b5faad3e02e1e12f38d, State: Initialized, Role: FOLLOWER
I20251212 08:54:11.322809 12432 consensus_queue.cc:260] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.323127 12432 raft_consensus.cc:399] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:11.323195 12432 raft_consensus.cc:493] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:11.323282 12432 raft_consensus.cc:3060] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:11.325692 12432 raft_consensus.cc:515] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.325888 12432 leader_election.cc:304] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dd7e4429e0d4b5faad3e02e1e12f38d; no voters:
I20251212 08:54:11.326320 12432 leader_election.cc:290] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:11.326455 12434 raft_consensus.cc:2804] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:11.326963 12432 ts_tablet_manager.cc:1434] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:54:11.327101 12434 raft_consensus.cc:697] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 LEADER]: Becoming Leader. State: Replica: 4dd7e4429e0d4b5faad3e02e1e12f38d, State: Running, Role: LEADER
I20251212 08:54:11.327210 12432 tablet_bootstrap.cc:492] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d: Bootstrap starting.
I20251212 08:54:11.327349 12434 consensus_queue.cc:237] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.328718 12432 tablet_bootstrap.cc:654] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:11.329552 12277 catalog_manager.cc:5654] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dd7e4429e0d4b5faad3e02e1e12f38d (127.0.208.1). New cstate: current_term: 1 leader_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:11.330360 12432 tablet_bootstrap.cc:492] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d: No bootstrap required, opened a new log
I20251212 08:54:11.330487 12432 ts_tablet_manager.cc:1403] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:11.331005 12432 raft_consensus.cc:359] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.331132 12432 raft_consensus.cc:385] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:11.331192 12432 raft_consensus.cc:740] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd7e4429e0d4b5faad3e02e1e12f38d, State: Initialized, Role: FOLLOWER
I20251212 08:54:11.331386 12432 consensus_queue.cc:260] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.331498 12432 raft_consensus.cc:399] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:11.331550 12432 raft_consensus.cc:493] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:11.331626 12432 raft_consensus.cc:3060] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:11.334425 12432 raft_consensus.cc:515] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.334642 12432 leader_election.cc:304] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dd7e4429e0d4b5faad3e02e1e12f38d; no voters:
I20251212 08:54:11.334826 12432 leader_election.cc:290] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:11.335237 12432 ts_tablet_manager.cc:1434] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:54:11.335479 12436 raft_consensus.cc:2804] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:11.335620 12436 raft_consensus.cc:697] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 LEADER]: Becoming Leader. State: Replica: 4dd7e4429e0d4b5faad3e02e1e12f38d, State: Running, Role: LEADER
I20251212 08:54:11.335836 12436 consensus_queue.cc:237] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.340538 12277 catalog_manager.cc:5654] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dd7e4429e0d4b5faad3e02e1e12f38d (127.0.208.1). New cstate: current_term: 1 leader_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:11.349170 12277 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47090:
name: "table_ea_flush_FLOAT"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "arr_0"
type: NESTED
is_key: false
is_nullable: true
encoding: PLAIN_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
nested_type {
array {
type: FLOAT
}
}
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000""\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:11.356045 12381 tablet_service.cc:1505] Processing CreateTablet for tablet 5fa506438a5447d0a9fac6cc5be86c3e (DEFAULT_TABLE table=table_ea_flush_FLOAT [id=2d39a693dafe43ae898ffefb9dfbda1d]), partition=RANGE (key) PARTITION 0 <= VALUES < 100
I20251212 08:54:11.356571 12381 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5fa506438a5447d0a9fac6cc5be86c3e. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:11.362390 12432 tablet_bootstrap.cc:492] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: Bootstrap starting.
I20251212 08:54:11.363777 12432 tablet_bootstrap.cc:654] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:11.365623 12432 tablet_bootstrap.cc:492] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: No bootstrap required, opened a new log
I20251212 08:54:11.365773 12432 ts_tablet_manager.cc:1403] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:11.366573 12432 raft_consensus.cc:359] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.366716 12432 raft_consensus.cc:385] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:11.366772 12432 raft_consensus.cc:740] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dd7e4429e0d4b5faad3e02e1e12f38d, State: Initialized, Role: FOLLOWER
I20251212 08:54:11.367026 12432 consensus_queue.cc:260] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.367173 12432 raft_consensus.cc:399] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:11.367214 12432 raft_consensus.cc:493] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:11.367270 12432 raft_consensus.cc:3060] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:11.369853 12432 raft_consensus.cc:515] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.370107 12432 leader_election.cc:304] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4dd7e4429e0d4b5faad3e02e1e12f38d; no voters:
I20251212 08:54:11.370299 12432 leader_election.cc:290] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:11.370394 12436 raft_consensus.cc:2804] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:11.370716 12436 raft_consensus.cc:697] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 LEADER]: Becoming Leader. State: Replica: 4dd7e4429e0d4b5faad3e02e1e12f38d, State: Running, Role: LEADER
I20251212 08:54:11.370719 12432 ts_tablet_manager.cc:1434] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20251212 08:54:11.371022 12436 consensus_queue.cc:237] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } }
I20251212 08:54:11.373310 12277 catalog_manager.cc:5654] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4dd7e4429e0d4b5faad3e02e1e12f38d (127.0.208.1). New cstate: current_term: 1 leader_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4dd7e4429e0d4b5faad3e02e1e12f38d" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 43253 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:11.412632 12277 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47090:
table { table_name: "table_ea_flush_FLOAT" } modify_external_catalogs: true
I20251212 08:54:11.412869 12277 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47090:
table { table_name: "table_ea_flush_FLOAT" } modify_external_catalogs: true
I20251212 08:54:11.415658 12277 catalog_manager.cc:5941] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d: Sending DeleteTablet for 1 replicas of tablet 5fa506438a5447d0a9fac6cc5be86c3e
I20251212 08:54:11.416384 12381 tablet_service.cc:1552] Processing DeleteTablet for tablet 5fa506438a5447d0a9fac6cc5be86c3e with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:54:11 UTC) from {username='slave'} at 127.0.0.1:39984
I20251212 08:54:11.416996 12445 tablet_replica.cc:333] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: stopping tablet replica
I20251212 08:54:11.417218 12445 raft_consensus.cc:2243] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.417436 12445 raft_consensus.cc:2272] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.418581 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:11.420394 12445 ts_tablet_manager.cc:1916] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:54:11.436388 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:11.441243 12445 ts_tablet_manager.cc:1929] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251212 08:54:11.441365 12445 log.cc:1199] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_29.1765529557162900-832-0/minicluster-data/ts-0-root/wals/5fa506438a5447d0a9fac6cc5be86c3e
I20251212 08:54:11.441676 12445 ts_tablet_manager.cc:1950] T 5fa506438a5447d0a9fac6cc5be86c3e P 4dd7e4429e0d4b5faad3e02e1e12f38d: Deleting consensus metadata
I20251212 08:54:11.442471 832 tablet_replica.cc:333] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d: stopping tablet replica
W20251212 08:54:11.442435 12264 catalog_manager.cc:4977] TS 4dd7e4429e0d4b5faad3e02e1e12f38d (127.0.208.1:43253): delete failed for tablet 5fa506438a5447d0a9fac6cc5be86c3e with error code TABLET_NOT_RUNNING: Service unavailable: Tablet Manager is not running: MANAGER_QUIESCING
I20251212 08:54:11.442653 832 raft_consensus.cc:2243] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.442835 832 raft_consensus.cc:2272] T e1203ea820394161a9805e4d38a143ef P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.443773 832 tablet_replica.cc:333] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d: stopping tablet replica
I20251212 08:54:11.443938 832 raft_consensus.cc:2243] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.444106 832 raft_consensus.cc:2272] T 5e2f4f86ed7a49a7bea53f37df2f0927 P 4dd7e4429e0d4b5faad3e02e1e12f38d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.459529 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
W20251212 08:54:11.461990 12264 catalog_manager.cc:4712] TS 4dd7e4429e0d4b5faad3e02e1e12f38d (127.0.208.1:43253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5fa506438a5447d0a9fac6cc5be86c3e: Network error: Client connection negotiation failed: client connection to 127.0.208.1:43253: connect: Connection refused (error 111)
I20251212 08:54:11.472396 832 master.cc:561] Master@127.0.208.62:33025 shutting down...
I20251212 08:54:11.518872 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.519167 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.519407 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P daa51d87af584c01bed44664fd1d304d: stopping tablet replica
I20251212 08:54:11.533583 832 master.cc:583] Master@127.0.208.62:33025 shutdown complete.
[ OK ] Params/ArrayColumnParamTest.EmptyAndNullArrays/29 (405 ms)
[ RUN ] Params/ArrayColumnParamTest.EmptyAndNullArrays/37
I20251212 08:54:11.544085 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:44995
I20251212 08:54:11.544633 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:11.547065 12450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.547458 12451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.547503 12453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:11.547899 832 server_base.cc:1047] running on GCE node
I20251212 08:54:11.548152 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:11.548203 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:11.548226 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529651548223 us; error 0 us; skew 500 ppm
I20251212 08:54:11.548394 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:11.549672 832 webserver.cc:492] Webserver started at http://127.0.208.62:37329/ using document root <none> and password file <none>
I20251212 08:54:11.549886 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:11.549971 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:11.550097 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:11.550710 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "993e9e499d994be2b81c716623fbe8f5"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.553341 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:11.554920 12458 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:11.555321 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20251212 08:54:11.555416 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "993e9e499d994be2b81c716623fbe8f5"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.555509 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:11.561367 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:11.561789 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:11.573642 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:44995
I20251212 08:54:11.573663 12519 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:44995 every 8 connection(s)
I20251212 08:54:11.575069 12520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:11.580488 12520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5: Bootstrap starting.
I20251212 08:54:11.581739 12520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:11.583245 12520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5: No bootstrap required, opened a new log
I20251212 08:54:11.583837 12520 raft_consensus.cc:359] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e9e499d994be2b81c716623fbe8f5" member_type: VOTER }
I20251212 08:54:11.583962 12520 raft_consensus.cc:385] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:11.584038 12520 raft_consensus.cc:740] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 993e9e499d994be2b81c716623fbe8f5, State: Initialized, Role: FOLLOWER
I20251212 08:54:11.584260 12520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e9e499d994be2b81c716623fbe8f5" member_type: VOTER }
I20251212 08:54:11.584435 12520 raft_consensus.cc:399] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:11.584502 12520 raft_consensus.cc:493] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:11.584582 12520 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:11.586709 12520 raft_consensus.cc:515] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e9e499d994be2b81c716623fbe8f5" member_type: VOTER }
I20251212 08:54:11.586899 12520 leader_election.cc:304] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 993e9e499d994be2b81c716623fbe8f5; no voters:
I20251212 08:54:11.587276 12520 leader_election.cc:290] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:11.587447 12523 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:11.587875 12523 raft_consensus.cc:697] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 1 LEADER]: Becoming Leader. State: Replica: 993e9e499d994be2b81c716623fbe8f5, State: Running, Role: LEADER
I20251212 08:54:11.588132 12520 sys_catalog.cc:565] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:11.588114 12523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e9e499d994be2b81c716623fbe8f5" member_type: VOTER }
I20251212 08:54:11.589025 12524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "993e9e499d994be2b81c716623fbe8f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e9e499d994be2b81c716623fbe8f5" member_type: VOTER } }
I20251212 08:54:11.589061 12525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 993e9e499d994be2b81c716623fbe8f5. Latest consensus state: current_term: 1 leader_uuid: "993e9e499d994be2b81c716623fbe8f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "993e9e499d994be2b81c716623fbe8f5" member_type: VOTER } }
I20251212 08:54:11.589284 12525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:11.590025 12524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:11.590543 12528 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:11.591738 12528 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:11.594592 12528 catalog_manager.cc:1357] Generated new cluster ID: d32d7bd3734b4c949dcc82ee36d681d7
I20251212 08:54:11.594669 12528 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:11.596120 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:11.609210 12528 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:11.609946 12528 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:11.621629 12528 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5: Generated new TSK 0
I20251212 08:54:11.621857 12528 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:11.628778 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:11.631279 12541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.631484 12542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.631983 12544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:11.632174 832 server_base.cc:1047] running on GCE node
I20251212 08:54:11.632445 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:11.632499 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:11.632522 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529651632519 us; error 0 us; skew 500 ppm
I20251212 08:54:11.632642 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:11.633742 832 webserver.cc:492] Webserver started at http://127.0.208.1:44247/ using document root <none> and password file <none>
I20251212 08:54:11.633913 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:11.633967 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:11.634066 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:11.634682 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "843cce12ea684cc1890569b02312fc03"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.637154 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20251212 08:54:11.638728 12549 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:11.639123 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:11.639231 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "843cce12ea684cc1890569b02312fc03"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.639341 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:11.653177 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:11.653692 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:11.655233 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:11.655303 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:11.655388 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:11.655444 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:11.671187 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:37533
I20251212 08:54:11.671202 12619 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:37533 every 8 connection(s)
I20251212 08:54:11.672887 12620 heartbeater.cc:344] Connected to a master server at 127.0.208.62:44995
I20251212 08:54:11.672994 12620 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:11.673254 12620 heartbeater.cc:507] Master 127.0.208.62:44995 requested a full tablet report, sending...
I20251212 08:54:11.674036 12475 ts_manager.cc:194] Registered new tserver with Master: 843cce12ea684cc1890569b02312fc03 (127.0.208.1:37533)
I20251212 08:54:11.674477 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002302128s
I20251212 08:54:11.675032 12475 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:56052
I20251212 08:54:11.684194 12475 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56064:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:11.686163 12620 heartbeater.cc:499] Master 127.0.208.62:44995 was elected leader, sending a full tablet report...
I20251212 08:54:11.697610 12580 tablet_service.cc:1505] Processing CreateTablet for tablet 64872036caff4e6499d8e3d3e625a753 (DEFAULT_TABLE table=client-testtb [id=860e8d82011a4f52b5344aa08936f6ff]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:11.697615 12579 tablet_service.cc:1505] Processing CreateTablet for tablet 07077b267f8242c1aa979dcc865e8b1b (DEFAULT_TABLE table=client-testtb [id=860e8d82011a4f52b5344aa08936f6ff]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:11.698155 12580 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64872036caff4e6499d8e3d3e625a753. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:11.698318 12579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 07077b267f8242c1aa979dcc865e8b1b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:11.706650 12630 tablet_bootstrap.cc:492] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03: Bootstrap starting.
I20251212 08:54:11.707878 12630 tablet_bootstrap.cc:654] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:11.709518 12630 tablet_bootstrap.cc:492] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03: No bootstrap required, opened a new log
I20251212 08:54:11.709668 12630 ts_tablet_manager.cc:1403] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:11.710400 12630 raft_consensus.cc:359] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.710538 12630 raft_consensus.cc:385] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:11.710608 12630 raft_consensus.cc:740] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843cce12ea684cc1890569b02312fc03, State: Initialized, Role: FOLLOWER
I20251212 08:54:11.710808 12630 consensus_queue.cc:260] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.710949 12630 raft_consensus.cc:399] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:11.711019 12630 raft_consensus.cc:493] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:11.711122 12630 raft_consensus.cc:3060] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:11.713308 12630 raft_consensus.cc:515] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.713523 12630 leader_election.cc:304] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 843cce12ea684cc1890569b02312fc03; no voters:
I20251212 08:54:11.713968 12630 leader_election.cc:290] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:11.714124 12632 raft_consensus.cc:2804] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:11.714573 12630 ts_tablet_manager.cc:1434] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:54:11.714857 12630 tablet_bootstrap.cc:492] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03: Bootstrap starting.
I20251212 08:54:11.714883 12632 raft_consensus.cc:697] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 1 LEADER]: Becoming Leader. State: Replica: 843cce12ea684cc1890569b02312fc03, State: Running, Role: LEADER
I20251212 08:54:11.715276 12632 consensus_queue.cc:237] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.716310 12630 tablet_bootstrap.cc:654] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:11.717645 12474 catalog_manager.cc:5654] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 843cce12ea684cc1890569b02312fc03 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "843cce12ea684cc1890569b02312fc03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:11.717901 12630 tablet_bootstrap.cc:492] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03: No bootstrap required, opened a new log
I20251212 08:54:11.718027 12630 ts_tablet_manager.cc:1403] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:11.718576 12630 raft_consensus.cc:359] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.718696 12630 raft_consensus.cc:385] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:11.718765 12630 raft_consensus.cc:740] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843cce12ea684cc1890569b02312fc03, State: Initialized, Role: FOLLOWER
I20251212 08:54:11.718967 12630 consensus_queue.cc:260] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.719120 12630 raft_consensus.cc:399] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:11.719178 12630 raft_consensus.cc:493] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:11.719267 12630 raft_consensus.cc:3060] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:11.721812 12630 raft_consensus.cc:515] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.721997 12630 leader_election.cc:304] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 843cce12ea684cc1890569b02312fc03; no voters:
I20251212 08:54:11.722157 12630 leader_election.cc:290] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:11.722237 12632 raft_consensus.cc:2804] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:11.722391 12632 raft_consensus.cc:697] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 1 LEADER]: Becoming Leader. State: Replica: 843cce12ea684cc1890569b02312fc03, State: Running, Role: LEADER
I20251212 08:54:11.722556 12630 ts_tablet_manager.cc:1434] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:54:11.722615 12632 consensus_queue.cc:237] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.724493 12474 catalog_manager.cc:5654] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 843cce12ea684cc1890569b02312fc03 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "843cce12ea684cc1890569b02312fc03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:11.734875 12474 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56064:
name: "table_ea_flush_UNIXTIME_MICROS"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "arr_0"
type: NESTED
is_key: false
is_nullable: true
encoding: PLAIN_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
nested_type {
array {
type: UNIXTIME_MICROS
}
}
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000""\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:11.741320 12579 tablet_service.cc:1505] Processing CreateTablet for tablet 973d4f913f8a4d2eb1d8aff40e2e77ff (DEFAULT_TABLE table=table_ea_flush_UNIXTIME_MICROS [id=48ae62835789497199e43051d68b3352]), partition=RANGE (key) PARTITION 0 <= VALUES < 100
I20251212 08:54:11.741770 12579 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 973d4f913f8a4d2eb1d8aff40e2e77ff. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:11.746883 12630 tablet_bootstrap.cc:492] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: Bootstrap starting.
I20251212 08:54:11.748085 12630 tablet_bootstrap.cc:654] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:11.749791 12630 tablet_bootstrap.cc:492] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: No bootstrap required, opened a new log
I20251212 08:54:11.750010 12630 ts_tablet_manager.cc:1403] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:11.750835 12630 raft_consensus.cc:359] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.750994 12630 raft_consensus.cc:385] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:11.751073 12630 raft_consensus.cc:740] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 843cce12ea684cc1890569b02312fc03, State: Initialized, Role: FOLLOWER
I20251212 08:54:11.751292 12630 consensus_queue.cc:260] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.751425 12630 raft_consensus.cc:399] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:11.751505 12630 raft_consensus.cc:493] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:11.751605 12630 raft_consensus.cc:3060] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:11.753851 12630 raft_consensus.cc:515] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.754086 12630 leader_election.cc:304] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 843cce12ea684cc1890569b02312fc03; no voters:
I20251212 08:54:11.754303 12630 leader_election.cc:290] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:11.754390 12632 raft_consensus.cc:2804] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:11.754535 12632 raft_consensus.cc:697] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 1 LEADER]: Becoming Leader. State: Replica: 843cce12ea684cc1890569b02312fc03, State: Running, Role: LEADER
I20251212 08:54:11.754804 12630 ts_tablet_manager.cc:1434] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20251212 08:54:11.754911 12632 consensus_queue.cc:237] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } }
I20251212 08:54:11.757026 12474 catalog_manager.cc:5654] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 843cce12ea684cc1890569b02312fc03 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "843cce12ea684cc1890569b02312fc03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "843cce12ea684cc1890569b02312fc03" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 37533 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:11.791419 12474 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56064:
table { table_name: "table_ea_flush_UNIXTIME_MICROS" } modify_external_catalogs: true
I20251212 08:54:11.791600 12474 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56064:
table { table_name: "table_ea_flush_UNIXTIME_MICROS" } modify_external_catalogs: true
I20251212 08:54:11.794375 12474 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5: Sending DeleteTablet for 1 replicas of tablet 973d4f913f8a4d2eb1d8aff40e2e77ff
I20251212 08:54:11.795047 12579 tablet_service.cc:1552] Processing DeleteTablet for tablet 973d4f913f8a4d2eb1d8aff40e2e77ff with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:54:11 UTC) from {username='slave'} at 127.0.0.1:43574
I20251212 08:54:11.795694 12643 tablet_replica.cc:333] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: stopping tablet replica
I20251212 08:54:11.795962 12643 raft_consensus.cc:2243] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.796192 12643 raft_consensus.cc:2272] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.796330 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:11.798173 12643 ts_tablet_manager.cc:1916] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:54:11.805071 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:11.808625 12643 ts_tablet_manager.cc:1929] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251212 08:54:11.808724 12643 log.cc:1199] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_37.1765529557162900-832-0/minicluster-data/ts-0-root/wals/973d4f913f8a4d2eb1d8aff40e2e77ff
I20251212 08:54:11.808979 12643 ts_tablet_manager.cc:1950] T 973d4f913f8a4d2eb1d8aff40e2e77ff P 843cce12ea684cc1890569b02312fc03: Deleting consensus metadata
I20251212 08:54:11.809887 832 tablet_replica.cc:333] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03: stopping tablet replica
I20251212 08:54:11.810031 832 raft_consensus.cc:2243] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.810170 832 raft_consensus.cc:2272] T 07077b267f8242c1aa979dcc865e8b1b P 843cce12ea684cc1890569b02312fc03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.810964 832 tablet_replica.cc:333] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03: stopping tablet replica
I20251212 08:54:11.811079 832 raft_consensus.cc:2243] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.811185 832 raft_consensus.cc:2272] T 64872036caff4e6499d8e3d3e625a753 P 843cce12ea684cc1890569b02312fc03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.826320 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:11.841164 832 master.cc:561] Master@127.0.208.62:44995 shutting down...
I20251212 08:54:11.914387 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:11.914700 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:11.914894 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 993e9e499d994be2b81c716623fbe8f5: stopping tablet replica
I20251212 08:54:11.929612 832 master.cc:583] Master@127.0.208.62:44995 shutdown complete.
[ OK ] Params/ArrayColumnParamTest.EmptyAndNullArrays/37 (396 ms)
[ RUN ] Params/ArrayColumnParamTest.EmptyAndNullArrays/45
I20251212 08:54:11.941480 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:41143
I20251212 08:54:11.942003 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:11.944454 12649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.944670 12651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:11.944670 12648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:11.944846 832 server_base.cc:1047] running on GCE node
I20251212 08:54:11.945225 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:11.945293 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:11.945317 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529651945313 us; error 0 us; skew 500 ppm
I20251212 08:54:11.945425 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:11.946532 832 webserver.cc:492] Webserver started at http://127.0.208.62:39047/ using document root <none> and password file <none>
I20251212 08:54:11.946704 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:11.946755 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:11.946852 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:11.947410 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "cf1aeb7925184fd68ada1e4ccefcdb05"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.949936 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20251212 08:54:11.951436 12656 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:11.951807 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:11.951915 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "cf1aeb7925184fd68ada1e4ccefcdb05"
format_stamp: "Formatted at 2025-12-12 08:54:11 on dist-test-slave-qdpb"
I20251212 08:54:11.952030 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:11.957554 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:11.957988 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:11.972815 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:41143
I20251212 08:54:11.972831 12717 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:41143 every 8 connection(s)
I20251212 08:54:11.974149 12718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:11.980515 12718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05: Bootstrap starting.
I20251212 08:54:11.981707 12718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:11.983261 12718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05: No bootstrap required, opened a new log
I20251212 08:54:11.983924 12718 raft_consensus.cc:359] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1aeb7925184fd68ada1e4ccefcdb05" member_type: VOTER }
I20251212 08:54:11.984047 12718 raft_consensus.cc:385] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:11.984082 12718 raft_consensus.cc:740] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf1aeb7925184fd68ada1e4ccefcdb05, State: Initialized, Role: FOLLOWER
I20251212 08:54:11.984270 12718 consensus_queue.cc:260] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1aeb7925184fd68ada1e4ccefcdb05" member_type: VOTER }
I20251212 08:54:11.984401 12718 raft_consensus.cc:399] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:11.984463 12718 raft_consensus.cc:493] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:11.984539 12718 raft_consensus.cc:3060] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:11.986614 12718 raft_consensus.cc:515] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1aeb7925184fd68ada1e4ccefcdb05" member_type: VOTER }
I20251212 08:54:11.986789 12718 leader_election.cc:304] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cf1aeb7925184fd68ada1e4ccefcdb05; no voters:
I20251212 08:54:11.987179 12718 leader_election.cc:290] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:11.987305 12721 raft_consensus.cc:2804] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:11.987746 12721 raft_consensus.cc:697] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 1 LEADER]: Becoming Leader. State: Replica: cf1aeb7925184fd68ada1e4ccefcdb05, State: Running, Role: LEADER
I20251212 08:54:11.988029 12718 sys_catalog.cc:565] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:11.988003 12721 consensus_queue.cc:237] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1aeb7925184fd68ada1e4ccefcdb05" member_type: VOTER }
I20251212 08:54:11.989022 12722 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cf1aeb7925184fd68ada1e4ccefcdb05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1aeb7925184fd68ada1e4ccefcdb05" member_type: VOTER } }
I20251212 08:54:11.989077 12723 sys_catalog.cc:455] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cf1aeb7925184fd68ada1e4ccefcdb05. Latest consensus state: current_term: 1 leader_uuid: "cf1aeb7925184fd68ada1e4ccefcdb05" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cf1aeb7925184fd68ada1e4ccefcdb05" member_type: VOTER } }
I20251212 08:54:11.989362 12723 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:11.989321 12722 sys_catalog.cc:458] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:11.990535 12727 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:11.991962 12727 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:11.993175 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:11.994801 12727 catalog_manager.cc:1357] Generated new cluster ID: 7daacb6bcaf54a82bfaeee5598b20cc2
I20251212 08:54:11.994884 12727 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:12.022200 12727 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:12.023020 12727 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:12.033808 12727 catalog_manager.cc:6027] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05: Generated new TSK 0
I20251212 08:54:12.034122 12727 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:12.058218 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:12.060814 12739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:12.061185 12742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:12.061252 12740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:12.061379 832 server_base.cc:1047] running on GCE node
I20251212 08:54:12.061681 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:12.061741 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:12.061779 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529652061774 us; error 0 us; skew 500 ppm
I20251212 08:54:12.061930 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:12.063066 832 webserver.cc:492] Webserver started at http://127.0.208.1:37691/ using document root <none> and password file <none>
I20251212 08:54:12.063263 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:12.063333 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:12.063452 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:12.064077 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "76317b0058e942dfa67a9e8606bf6b3e"
format_stamp: "Formatted at 2025-12-12 08:54:12 on dist-test-slave-qdpb"
I20251212 08:54:12.066548 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251212 08:54:12.068073 12747 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:12.068432 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20251212 08:54:12.068533 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "76317b0058e942dfa67a9e8606bf6b3e"
format_stamp: "Formatted at 2025-12-12 08:54:12 on dist-test-slave-qdpb"
I20251212 08:54:12.068637 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:12.086761 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:12.087277 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:12.088960 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:12.089049 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:12.089154 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:12.089221 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:12.105854 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:40007
I20251212 08:54:12.105863 12817 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:40007 every 8 connection(s)
I20251212 08:54:12.107721 12818 heartbeater.cc:344] Connected to a master server at 127.0.208.62:41143
I20251212 08:54:12.107857 12818 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:12.108198 12818 heartbeater.cc:507] Master 127.0.208.62:41143 requested a full tablet report, sending...
I20251212 08:54:12.109077 12673 ts_manager.cc:194] Registered new tserver with Master: 76317b0058e942dfa67a9e8606bf6b3e (127.0.208.1:40007)
I20251212 08:54:12.109491 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002328309s
I20251212 08:54:12.110129 12673 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33180
I20251212 08:54:12.119894 12673 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33196:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:12.121488 12818 heartbeater.cc:499] Master 127.0.208.62:41143 was elected leader, sending a full tablet report...
I20251212 08:54:12.133395 12777 tablet_service.cc:1505] Processing CreateTablet for tablet a52288ae6aef4d429b3c17a193d8a441 (DEFAULT_TABLE table=client-testtb [id=09b458ec39714109bf14c898590d5c36]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:12.133395 12778 tablet_service.cc:1505] Processing CreateTablet for tablet 758edd08b7f64e47a73148922b865faa (DEFAULT_TABLE table=client-testtb [id=09b458ec39714109bf14c898590d5c36]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:12.133913 12777 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a52288ae6aef4d429b3c17a193d8a441. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:12.134095 12778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 758edd08b7f64e47a73148922b865faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:12.139652 12828 tablet_bootstrap.cc:492] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e: Bootstrap starting.
I20251212 08:54:12.140875 12828 tablet_bootstrap.cc:654] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:12.142375 12828 tablet_bootstrap.cc:492] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e: No bootstrap required, opened a new log
I20251212 08:54:12.142478 12828 ts_tablet_manager.cc:1403] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:12.142992 12828 raft_consensus.cc:359] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.143114 12828 raft_consensus.cc:385] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:12.143147 12828 raft_consensus.cc:740] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76317b0058e942dfa67a9e8606bf6b3e, State: Initialized, Role: FOLLOWER
I20251212 08:54:12.143296 12828 consensus_queue.cc:260] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.143394 12828 raft_consensus.cc:399] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:12.143429 12828 raft_consensus.cc:493] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:12.143482 12828 raft_consensus.cc:3060] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:12.145555 12828 raft_consensus.cc:515] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.145740 12828 leader_election.cc:304] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76317b0058e942dfa67a9e8606bf6b3e; no voters:
I20251212 08:54:12.146227 12828 leader_election.cc:290] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:12.146329 12830 raft_consensus.cc:2804] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:12.146719 12830 raft_consensus.cc:697] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 1 LEADER]: Becoming Leader. State: Replica: 76317b0058e942dfa67a9e8606bf6b3e, State: Running, Role: LEADER
I20251212 08:54:12.147032 12830 consensus_queue.cc:237] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.148362 12828 ts_tablet_manager.cc:1434] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20251212 08:54:12.148639 12828 tablet_bootstrap.cc:492] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e: Bootstrap starting.
I20251212 08:54:12.149097 12672 catalog_manager.cc:5654] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76317b0058e942dfa67a9e8606bf6b3e (127.0.208.1). New cstate: current_term: 1 leader_uuid: "76317b0058e942dfa67a9e8606bf6b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:12.150735 12828 tablet_bootstrap.cc:654] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:12.153132 12828 tablet_bootstrap.cc:492] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e: No bootstrap required, opened a new log
I20251212 08:54:12.153273 12828 ts_tablet_manager.cc:1403] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20251212 08:54:12.153987 12828 raft_consensus.cc:359] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.154132 12828 raft_consensus.cc:385] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:12.154202 12828 raft_consensus.cc:740] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76317b0058e942dfa67a9e8606bf6b3e, State: Initialized, Role: FOLLOWER
I20251212 08:54:12.154412 12828 consensus_queue.cc:260] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.154532 12828 raft_consensus.cc:399] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:12.154596 12828 raft_consensus.cc:493] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:12.154675 12828 raft_consensus.cc:3060] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:12.157029 12828 raft_consensus.cc:515] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.157219 12828 leader_election.cc:304] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76317b0058e942dfa67a9e8606bf6b3e; no voters:
I20251212 08:54:12.157373 12828 leader_election.cc:290] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:12.157433 12831 raft_consensus.cc:2804] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:12.157555 12831 raft_consensus.cc:697] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 LEADER]: Becoming Leader. State: Replica: 76317b0058e942dfa67a9e8606bf6b3e, State: Running, Role: LEADER
I20251212 08:54:12.157789 12828 ts_tablet_manager.cc:1434] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20251212 08:54:12.157744 12831 consensus_queue.cc:237] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.159700 12672 catalog_manager.cc:5654] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76317b0058e942dfa67a9e8606bf6b3e (127.0.208.1). New cstate: current_term: 1 leader_uuid: "76317b0058e942dfa67a9e8606bf6b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:12.171769 12672 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33196:
name: "table_ea_flush_DECIMAL32"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "arr_0"
type: NESTED
is_key: false
is_nullable: true
encoding: PLAIN_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
type_attributes {
precision: 6
scale: 2
}
immutable: false
nested_type {
array {
type: DECIMAL32
}
}
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000""\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:12.178100 12778 tablet_service.cc:1505] Processing CreateTablet for tablet e4f5c001177147108d76d7d10f454646 (DEFAULT_TABLE table=table_ea_flush_DECIMAL32 [id=b981a8de73b04fceb4ba2c4b2ddf4c22]), partition=RANGE (key) PARTITION 0 <= VALUES < 100
I20251212 08:54:12.178642 12778 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e4f5c001177147108d76d7d10f454646. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:12.183912 12828 tablet_bootstrap.cc:492] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: Bootstrap starting.
I20251212 08:54:12.185150 12828 tablet_bootstrap.cc:654] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:12.186757 12828 tablet_bootstrap.cc:492] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: No bootstrap required, opened a new log
I20251212 08:54:12.186890 12828 ts_tablet_manager.cc:1403] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:12.187669 12828 raft_consensus.cc:359] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.187852 12828 raft_consensus.cc:385] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:12.187924 12828 raft_consensus.cc:740] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76317b0058e942dfa67a9e8606bf6b3e, State: Initialized, Role: FOLLOWER
I20251212 08:54:12.188133 12828 consensus_queue.cc:260] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.188252 12828 raft_consensus.cc:399] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:12.188330 12828 raft_consensus.cc:493] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:12.188400 12828 raft_consensus.cc:3060] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:12.190719 12828 raft_consensus.cc:515] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.190910 12828 leader_election.cc:304] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76317b0058e942dfa67a9e8606bf6b3e; no voters:
I20251212 08:54:12.191061 12828 leader_election.cc:290] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:12.191160 12831 raft_consensus.cc:2804] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:12.191285 12831 raft_consensus.cc:697] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 LEADER]: Becoming Leader. State: Replica: 76317b0058e942dfa67a9e8606bf6b3e, State: Running, Role: LEADER
I20251212 08:54:12.191483 12828 ts_tablet_manager.cc:1434] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20251212 08:54:12.191520 12831 consensus_queue.cc:237] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } }
I20251212 08:54:12.193784 12672 catalog_manager.cc:5654] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e reported cstate change: term changed from 0 to 1, leader changed from <none> to 76317b0058e942dfa67a9e8606bf6b3e (127.0.208.1). New cstate: current_term: 1 leader_uuid: "76317b0058e942dfa67a9e8606bf6b3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76317b0058e942dfa67a9e8606bf6b3e" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 40007 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:12.227631 12673 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33196:
table { table_name: "table_ea_flush_DECIMAL32" } modify_external_catalogs: true
I20251212 08:54:12.227844 12673 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33196:
table { table_name: "table_ea_flush_DECIMAL32" } modify_external_catalogs: true
I20251212 08:54:12.230559 12673 catalog_manager.cc:5941] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05: Sending DeleteTablet for 1 replicas of tablet e4f5c001177147108d76d7d10f454646
I20251212 08:54:12.231180 12778 tablet_service.cc:1552] Processing DeleteTablet for tablet e4f5c001177147108d76d7d10f454646 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:54:12 UTC) from {username='slave'} at 127.0.0.1:43624
I20251212 08:54:12.231833 12841 tablet_replica.cc:333] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: stopping tablet replica
I20251212 08:54:12.232101 12841 raft_consensus.cc:2243] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:12.232357 12841 raft_consensus.cc:2272] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:12.233295 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:12.233394 12841 ts_tablet_manager.cc:1916] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:54:12.245775 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:12.246037 12841 ts_tablet_manager.cc:1929] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251212 08:54:12.246181 12841 log.cc:1199] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_45.1765529557162900-832-0/minicluster-data/ts-0-root/wals/e4f5c001177147108d76d7d10f454646
I20251212 08:54:12.246517 12841 ts_tablet_manager.cc:1950] T e4f5c001177147108d76d7d10f454646 P 76317b0058e942dfa67a9e8606bf6b3e: Deleting consensus metadata
I20251212 08:54:12.247174 832 tablet_replica.cc:333] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e: stopping tablet replica
I20251212 08:54:12.247359 832 raft_consensus.cc:2243] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:12.247522 832 raft_consensus.cc:2272] T 758edd08b7f64e47a73148922b865faa P 76317b0058e942dfa67a9e8606bf6b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:12.248396 832 tablet_replica.cc:333] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e: stopping tablet replica
I20251212 08:54:12.248538 832 raft_consensus.cc:2243] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:12.248682 832 raft_consensus.cc:2272] T a52288ae6aef4d429b3c17a193d8a441 P 76317b0058e942dfa67a9e8606bf6b3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:12.264211 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:12.279168 832 master.cc:561] Master@127.0.208.62:41143 shutting down...
I20251212 08:54:12.307891 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:12.308128 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:12.308326 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P cf1aeb7925184fd68ada1e4ccefcdb05: stopping tablet replica
I20251212 08:54:12.322742 832 master.cc:583] Master@127.0.208.62:41143 shutdown complete.
[ OK ] Params/ArrayColumnParamTest.EmptyAndNullArrays/45 (392 ms)
[ RUN ] Params/ArrayColumnParamTest.EmptyAndNullArrays/53
I20251212 08:54:12.333544 832 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.0.208.62:41053
I20251212 08:54:12.334033 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:12.336403 12845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:12.336593 12846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:12.337332 832 server_base.cc:1047] running on GCE node
W20251212 08:54:12.337647 12848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:12.337991 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:12.338049 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:12.338074 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529652338071 us; error 0 us; skew 500 ppm
I20251212 08:54:12.338243 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:12.339392 832 webserver.cc:492] Webserver started at http://127.0.208.62:38337/ using document root <none> and password file <none>
I20251212 08:54:12.339588 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:12.339650 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:12.339773 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:12.340433 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/master-0-root/instance:
uuid: "7d6c340a93d4421483df51c8ca34b4c2"
format_stamp: "Formatted at 2025-12-12 08:54:12 on dist-test-slave-qdpb"
I20251212 08:54:12.342869 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20251212 08:54:12.344471 12853 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:12.344831 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:12.344969 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/master-0-root
uuid: "7d6c340a93d4421483df51c8ca34b4c2"
format_stamp: "Formatted at 2025-12-12 08:54:12 on dist-test-slave-qdpb"
I20251212 08:54:12.345079 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:12.351399 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:12.351842 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:12.365551 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.62:41053
I20251212 08:54:12.365574 12914 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.62:41053 every 8 connection(s)
I20251212 08:54:12.366916 12915 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:12.372089 12915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2: Bootstrap starting.
I20251212 08:54:12.373652 12915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:12.375329 12915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2: No bootstrap required, opened a new log
I20251212 08:54:12.375968 12915 raft_consensus.cc:359] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6c340a93d4421483df51c8ca34b4c2" member_type: VOTER }
I20251212 08:54:12.376107 12915 raft_consensus.cc:385] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:12.376152 12915 raft_consensus.cc:740] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6c340a93d4421483df51c8ca34b4c2, State: Initialized, Role: FOLLOWER
I20251212 08:54:12.376385 12915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6c340a93d4421483df51c8ca34b4c2" member_type: VOTER }
I20251212 08:54:12.376505 12915 raft_consensus.cc:399] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:12.376547 12915 raft_consensus.cc:493] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:12.376601 12915 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:12.379055 12915 raft_consensus.cc:515] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6c340a93d4421483df51c8ca34b4c2" member_type: VOTER }
I20251212 08:54:12.379261 12915 leader_election.cc:304] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d6c340a93d4421483df51c8ca34b4c2; no voters:
I20251212 08:54:12.379647 12915 leader_election.cc:290] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:12.379817 12918 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:12.380245 12918 raft_consensus.cc:697] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 1 LEADER]: Becoming Leader. State: Replica: 7d6c340a93d4421483df51c8ca34b4c2, State: Running, Role: LEADER
I20251212 08:54:12.380512 12918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6c340a93d4421483df51c8ca34b4c2" member_type: VOTER }
I20251212 08:54:12.380610 12915 sys_catalog.cc:565] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [sys.catalog]: configured and running, proceeding with master startup.
I20251212 08:54:12.381356 12919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d6c340a93d4421483df51c8ca34b4c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6c340a93d4421483df51c8ca34b4c2" member_type: VOTER } }
I20251212 08:54:12.381435 12920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d6c340a93d4421483df51c8ca34b4c2. Latest consensus state: current_term: 1 leader_uuid: "7d6c340a93d4421483df51c8ca34b4c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6c340a93d4421483df51c8ca34b4c2" member_type: VOTER } }
I20251212 08:54:12.381510 12919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:12.381564 12920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [sys.catalog]: This master's current role is: LEADER
I20251212 08:54:12.382164 12923 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251212 08:54:12.383728 12923 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251212 08:54:12.385512 832 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251212 08:54:12.386698 12923 catalog_manager.cc:1357] Generated new cluster ID: d765809569c34129b95061922e2ff5b1
I20251212 08:54:12.386801 12923 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251212 08:54:12.397866 12923 catalog_manager.cc:1380] Generated new certificate authority record
I20251212 08:54:12.398711 12923 catalog_manager.cc:1514] Loading token signing keys...
I20251212 08:54:12.410215 12923 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2: Generated new TSK 0
I20251212 08:54:12.410488 12923 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251212 08:54:12.418346 832 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251212 08:54:12.421051 12936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:12.421092 12937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251212 08:54:12.421386 12939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251212 08:54:12.421586 832 server_base.cc:1047] running on GCE node
I20251212 08:54:12.421810 832 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251212 08:54:12.421860 832 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251212 08:54:12.421882 832 hybrid_clock.cc:648] HybridClock initialized: now 1765529652421880 us; error 0 us; skew 500 ppm
I20251212 08:54:12.421994 832 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251212 08:54:12.423171 832 webserver.cc:492] Webserver started at http://127.0.208.1:46333/ using document root <none> and password file <none>
I20251212 08:54:12.423346 832 fs_manager.cc:362] Metadata directory not provided
I20251212 08:54:12.423400 832 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251212 08:54:12.423489 832 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251212 08:54:12.424098 832 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/ts-0-root/instance:
uuid: "d335daaf5db7444585b1ed2147cbb9e9"
format_stamp: "Formatted at 2025-12-12 08:54:12 on dist-test-slave-qdpb"
I20251212 08:54:12.426604 832 fs_manager.cc:696] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20251212 08:54:12.428217 12944 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:12.428562 832 fs_manager.cc:730] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20251212 08:54:12.428658 832 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/ts-0-root
uuid: "d335daaf5db7444585b1ed2147cbb9e9"
format_stamp: "Formatted at 2025-12-12 08:54:12 on dist-test-slave-qdpb"
I20251212 08:54:12.428772 832 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251212 08:54:12.459323 832 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251212 08:54:12.459843 832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251212 08:54:12.461581 832 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251212 08:54:12.461668 832 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:12.461771 832 ts_tablet_manager.cc:616] Registered 0 tablets
I20251212 08:54:12.461843 832 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251212 08:54:12.478515 832 rpc_server.cc:307] RPC server started. Bound to: 127.0.208.1:38093
I20251212 08:54:12.478528 13014 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.0.208.1:38093 every 8 connection(s)
I20251212 08:54:12.480500 13015 heartbeater.cc:344] Connected to a master server at 127.0.208.62:41053
I20251212 08:54:12.480643 13015 heartbeater.cc:461] Registering TS with master...
I20251212 08:54:12.481024 13015 heartbeater.cc:507] Master 127.0.208.62:41053 requested a full tablet report, sending...
I20251212 08:54:12.481921 12870 ts_manager.cc:194] Registered new tserver with Master: d335daaf5db7444585b1ed2147cbb9e9 (127.0.208.1:38093)
I20251212 08:54:12.481983 832 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.002370779s
I20251212 08:54:12.483829 12870 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:53498
I20251212 08:54:12.493003 12870 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53526:
name: "client-testtb"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "non_null_with_default"
type: INT32
is_key: false
is_nullable: false
read_default_value: "90\000\000"
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\004\001\000\t\000\000\000""\004\001\000\t\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:12.495604 13015 heartbeater.cc:499] Master 127.0.208.62:41053 was elected leader, sending a full tablet report...
I20251212 08:54:12.505400 12975 tablet_service.cc:1505] Processing CreateTablet for tablet 0260eac6a30e4240a270759d26a3185d (DEFAULT_TABLE table=client-testtb [id=407ea6a3213748f89ca4d7c845097eee]), partition=RANGE (key) PARTITION VALUES < 9
I20251212 08:54:12.505424 12974 tablet_service.cc:1505] Processing CreateTablet for tablet ea096d2781f44fde9d4c8b21482144b5 (DEFAULT_TABLE table=client-testtb [id=407ea6a3213748f89ca4d7c845097eee]), partition=RANGE (key) PARTITION 9 <= VALUES
I20251212 08:54:12.505888 12975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0260eac6a30e4240a270759d26a3185d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:12.506085 12974 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea096d2781f44fde9d4c8b21482144b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:12.511690 13025 tablet_bootstrap.cc:492] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9: Bootstrap starting.
I20251212 08:54:12.512910 13025 tablet_bootstrap.cc:654] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:12.514395 13025 tablet_bootstrap.cc:492] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9: No bootstrap required, opened a new log
I20251212 08:54:12.514504 13025 ts_tablet_manager.cc:1403] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:12.515048 13025 raft_consensus.cc:359] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.515177 13025 raft_consensus.cc:385] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:12.515232 13025 raft_consensus.cc:740] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d335daaf5db7444585b1ed2147cbb9e9, State: Initialized, Role: FOLLOWER
I20251212 08:54:12.515517 13025 consensus_queue.cc:260] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.515650 13025 raft_consensus.cc:399] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:12.515715 13025 raft_consensus.cc:493] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:12.515786 13025 raft_consensus.cc:3060] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:12.517860 13025 raft_consensus.cc:515] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.518086 13025 leader_election.cc:304] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d335daaf5db7444585b1ed2147cbb9e9; no voters:
I20251212 08:54:12.518543 13025 leader_election.cc:290] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:12.518724 13027 raft_consensus.cc:2804] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:12.519235 13027 raft_consensus.cc:697] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 1 LEADER]: Becoming Leader. State: Replica: d335daaf5db7444585b1ed2147cbb9e9, State: Running, Role: LEADER
I20251212 08:54:12.519548 13027 consensus_queue.cc:237] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.519757 13025 ts_tablet_manager.cc:1434] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20251212 08:54:12.520015 13025 tablet_bootstrap.cc:492] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9: Bootstrap starting.
I20251212 08:54:12.521548 13025 tablet_bootstrap.cc:654] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:12.521456 12870 catalog_manager.cc:5654] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d335daaf5db7444585b1ed2147cbb9e9 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "d335daaf5db7444585b1ed2147cbb9e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:12.523065 13025 tablet_bootstrap.cc:492] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9: No bootstrap required, opened a new log
I20251212 08:54:12.523170 13025 ts_tablet_manager.cc:1403] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:12.523772 13025 raft_consensus.cc:359] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.523885 13025 raft_consensus.cc:385] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:12.523942 13025 raft_consensus.cc:740] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d335daaf5db7444585b1ed2147cbb9e9, State: Initialized, Role: FOLLOWER
I20251212 08:54:12.524142 13025 consensus_queue.cc:260] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.524303 13025 raft_consensus.cc:399] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:12.524376 13025 raft_consensus.cc:493] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:12.524441 13025 raft_consensus.cc:3060] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:12.526575 13025 raft_consensus.cc:515] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.526737 13025 leader_election.cc:304] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d335daaf5db7444585b1ed2147cbb9e9; no voters:
I20251212 08:54:12.526862 13025 leader_election.cc:290] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:12.526943 13027 raft_consensus.cc:2804] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:12.527055 13027 raft_consensus.cc:697] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 LEADER]: Becoming Leader. State: Replica: d335daaf5db7444585b1ed2147cbb9e9, State: Running, Role: LEADER
I20251212 08:54:12.527276 13025 ts_tablet_manager.cc:1434] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20251212 08:54:12.527257 13027 consensus_queue.cc:237] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.529240 12870 catalog_manager.cc:5654] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d335daaf5db7444585b1ed2147cbb9e9 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "d335daaf5db7444585b1ed2147cbb9e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:12.535084 12869 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53526:
name: "table_ea_flush_VARCHAR"
schema {
columns {
name: "key"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "arr_0"
type: NESTED
is_key: false
is_nullable: true
encoding: PLAIN_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
type_attributes {
length: 10
}
immutable: false
nested_type {
array {
type: VARCHAR
}
}
}
}
num_replicas: 1
split_rows_range_bounds {
rows: "\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000""\010\001\000\377\377\377\377\377\377\377\377\t\001\000c\000\000\000\000\000\000\000"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251212 08:54:12.541621 12975 tablet_service.cc:1505] Processing CreateTablet for tablet ad6fa98a590a4e6f911e631942de47b8 (DEFAULT_TABLE table=table_ea_flush_VARCHAR [id=746ed506b9ac41dc8331f4294a4e9853]), partition=RANGE (key) PARTITION 0 <= VALUES < 100
I20251212 08:54:12.542098 12975 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ad6fa98a590a4e6f911e631942de47b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20251212 08:54:12.547482 13025 tablet_bootstrap.cc:492] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: Bootstrap starting.
I20251212 08:54:12.548796 13025 tablet_bootstrap.cc:654] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: Neither blocks nor log segments found. Creating new log.
I20251212 08:54:12.550211 13025 tablet_bootstrap.cc:492] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: No bootstrap required, opened a new log
I20251212 08:54:12.550318 13025 ts_tablet_manager.cc:1403] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20251212 08:54:12.550822 13025 raft_consensus.cc:359] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.550974 13025 raft_consensus.cc:385] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251212 08:54:12.551039 13025 raft_consensus.cc:740] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d335daaf5db7444585b1ed2147cbb9e9, State: Initialized, Role: FOLLOWER
I20251212 08:54:12.551230 13025 consensus_queue.cc:260] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.551338 13025 raft_consensus.cc:399] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251212 08:54:12.551386 13025 raft_consensus.cc:493] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251212 08:54:12.551465 13025 raft_consensus.cc:3060] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 0 FOLLOWER]: Advancing to term 1
I20251212 08:54:12.553556 13025 raft_consensus.cc:515] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.553723 13025 leader_election.cc:304] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d335daaf5db7444585b1ed2147cbb9e9; no voters:
I20251212 08:54:12.553853 13025 leader_election.cc:290] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251212 08:54:12.553938 13027 raft_consensus.cc:2804] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 FOLLOWER]: Leader election won for term 1
I20251212 08:54:12.554050 13027 raft_consensus.cc:697] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 LEADER]: Becoming Leader. State: Replica: d335daaf5db7444585b1ed2147cbb9e9, State: Running, Role: LEADER
I20251212 08:54:12.554260 13025 ts_tablet_manager.cc:1434] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20251212 08:54:12.554283 13027 consensus_queue.cc:237] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } }
I20251212 08:54:12.556314 12870 catalog_manager.cc:5654] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d335daaf5db7444585b1ed2147cbb9e9 (127.0.208.1). New cstate: current_term: 1 leader_uuid: "d335daaf5db7444585b1ed2147cbb9e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d335daaf5db7444585b1ed2147cbb9e9" member_type: VOTER last_known_addr { host: "127.0.208.1" port: 38093 } health_report { overall_health: HEALTHY } } }
I20251212 08:54:12.593116 12870 catalog_manager.cc:2507] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53526:
table { table_name: "table_ea_flush_VARCHAR" } modify_external_catalogs: true
I20251212 08:54:12.593313 12870 catalog_manager.cc:2755] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53526:
table { table_name: "table_ea_flush_VARCHAR" } modify_external_catalogs: true
I20251212 08:54:12.596153 12870 catalog_manager.cc:5941] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2: Sending DeleteTablet for 1 replicas of tablet ad6fa98a590a4e6f911e631942de47b8
I20251212 08:54:12.596843 12975 tablet_service.cc:1552] Processing DeleteTablet for tablet ad6fa98a590a4e6f911e631942de47b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2025-12-12 08:54:12 UTC) from {username='slave'} at 127.0.0.1:41804
I20251212 08:54:12.597541 13038 tablet_replica.cc:333] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: stopping tablet replica
I20251212 08:54:12.597887 13038 raft_consensus.cc:2243] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:12.598024 832 tablet_server.cc:178] TabletServer@127.0.208.1:0 shutting down...
I20251212 08:54:12.598263 13038 raft_consensus.cc:2272] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:12.600164 13038 ts_tablet_manager.cc:1916] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20251212 08:54:12.620975 13038 ts_tablet_manager.cc:1929] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20251212 08:54:12.621173 13038 log.cc:1199] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: Deleting WAL directory at /tmp/dist-test-taskxqoanx/test-tmp/client-test.6.Params_ArrayColumnParamTest.EmptyAndNullArrays_53.1765529557162900-832-0/minicluster-data/ts-0-root/wals/ad6fa98a590a4e6f911e631942de47b8
I20251212 08:54:12.621632 13038 ts_tablet_manager.cc:1950] T ad6fa98a590a4e6f911e631942de47b8 P d335daaf5db7444585b1ed2147cbb9e9: Deleting consensus metadata
I20251212 08:54:12.622244 832 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251212 08:54:12.623183 832 tablet_replica.cc:333] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9: stopping tablet replica
I20251212 08:54:12.623198 12857 catalog_manager.cc:4985] TS d335daaf5db7444585b1ed2147cbb9e9 (127.0.208.1:38093): tablet ad6fa98a590a4e6f911e631942de47b8 (table table_ea_flush_VARCHAR [id=746ed506b9ac41dc8331f4294a4e9853]) successfully deleted
I20251212 08:54:12.623517 832 raft_consensus.cc:2243] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:12.623836 832 raft_consensus.cc:2272] T 0260eac6a30e4240a270759d26a3185d P d335daaf5db7444585b1ed2147cbb9e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:12.625128 832 tablet_replica.cc:333] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9: stopping tablet replica
I20251212 08:54:12.625365 832 raft_consensus.cc:2243] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:12.625617 832 raft_consensus.cc:2272] T ea096d2781f44fde9d4c8b21482144b5 P d335daaf5db7444585b1ed2147cbb9e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:12.631134 832 tablet_server.cc:195] TabletServer@127.0.208.1:0 shutdown complete.
I20251212 08:54:12.636072 832 master.cc:561] Master@127.0.208.62:41053 shutting down...
I20251212 08:54:12.653795 832 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 1 LEADER]: Raft consensus shutting down.
I20251212 08:54:12.654073 832 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251212 08:54:12.654266 832 tablet_replica.cc:333] T 00000000000000000000000000000000 P 7d6c340a93d4421483df51c8ca34b4c2: stopping tablet replica
I20251212 08:54:12.659147 832 master.cc:583] Master@127.0.208.62:41053 shutdown complete.
[ OK ] Params/ArrayColumnParamTest.EmptyAndNullArrays/53 (336 ms)
[----------] 7 tests from Params/ArrayColumnParamTest (2969 ms total)
[----------] Global test environment tear-down
[==========] 35 tests from 9 test suites ran. (95428 ms total)
[ PASSED ] 35 tests.
I20251212 08:54:12.675937 832 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 62 messages since previous log ~260 seconds ago
I20251212 08:54:12.676041 832 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 1 messages since previous log ~12 seconds ago
I20251212 08:54:12.676067 832 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 1 messages since previous log ~12 seconds ago
I20251212 08:54:12.676120 832 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/tablet/tablet.cc:2378: suppressed but not reported on 14 messages since previous log ~9 seconds ago
I20251212 08:54:12.676170 832 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:4712: suppressed but not reported on 4 messages since previous log ~1 seconds ago
I20251212 08:54:12.676208 832 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 8 messages since previous log ~2 seconds ago
I20251212 08:54:12.676249 832 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/master/catalog_manager.cc:4977: suppressed but not reported on 2 messages since previous log ~1 seconds ago
=================================================================
==832==ERROR: LeakSanitizer: detected memory leaks
Direct leak of 224 byte(s) in 1 object(s) allocated from:
#0 0x83b838 in operator new(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:99
#1 0x7f55c8c86f91 in std::_MakeUniq<kudu::rpc::RequestPayload>::__single_object std::make_unique<kudu::rpc::RequestPayload, kudu::rpc::RemoteMethod const&>(kudu::rpc::RemoteMethod const&) ../../../include/c++/7.5.0/bits/unique_ptr.h:821:30
#2 0x7f55c8c80580 in kudu::rpc::RequestPayload::CreateRequestPayload(kudu::rpc::RemoteMethod const&, google::protobuf::Message const&, std::vector<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> >, std::allocator<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> > > >&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:162:18
#3 0x7f55c8c94a3f in kudu::rpc::Proxy::AsyncRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:209:22
#4 0x7f55cb8469c9 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/consensus/consensus.proxy.cc:56:3
#5 0x7f55ccc0ceb0 in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:669:21
#6 0x7f55ccc065ed in kudu::consensus::Peer::SendNextRequest(bool, kudu::consensus::MultiRaftConsensusData*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:377:13
#7 0x7f55ccc10ae1 in kudu::consensus::Peer::SignalRequest(bool)::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:213:12
#8 0x7f55ccc107cd in std::_Function_handler<void (), kudu::consensus::Peer::SignalRequest(bool)::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#9 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#10 0x7f55c22af20f in kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7
#11 0x7f55c22b252b in kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48
#12 0x7f55c22b23a8 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_1>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#13 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#14 0x7f55c228e3f8 in kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3
#15 0x7f55c50fe6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Direct leak of 24 byte(s) in 1 object(s) allocated from:
#0 0x83b838 in operator new(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:99
#1 0x7f55c8c94400 in kudu::rpc::Proxy::RefreshDnsAndEnqueueRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::unique_ptr<kudu::rpc::RequestPayload, std::default_delete<kudu::rpc::RequestPayload> >, google::protobuf::Message*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:168:29
#2 0x7f55c8c97723 in kudu::rpc::Proxy::AsyncRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, std::function<void ()> const&)::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:244:7
#3 0x7f55c8c96ecd in std::_Function_handler<void (), kudu::rpc::Proxy::AsyncRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, std::function<void ()> const&)::$_1>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#4 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#5 0x7f55c8c82f1f in kudu::rpc::OutboundCall::CallCallback() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:308:5
#6 0x7f55c8c83da4 in kudu::rpc::OutboundCall::SetFailed(kudu::Status, kudu::rpc::OutboundCall::Phase, std::unique_ptr<kudu::rpc::ErrorStatusPB, std::default_delete<kudu::rpc::ErrorStatusPB> >) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:399:3
#7 0x7f55c8c17f22 in kudu::rpc::Connection::Shutdown(kudu::Status const&, std::unique_ptr<kudu::rpc::ErrorStatusPB, std::default_delete<kudu::rpc::ErrorStatusPB> >) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/connection.cc:175:16
#8 0x7f55c8c9e9ff in kudu::rpc::ReactorThread::ShutdownInternal() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/reactor.cc:262:11
#9 0x7f55c8ca030b in kudu::rpc::ReactorThread::AsyncHandler(ev::async&, int) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/reactor.cc:338:5
#10 0x7f55c8cb2584 in void ev::base<ev_async, ev::async>::method_thunk<kudu::rpc::ReactorThread, &(kudu::rpc::ReactorThread::AsyncHandler(ev::async&, int))>(ev_loop*, ev_async*, int) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/uninstrumented/include/ev++.h:497:7
#11 0x7f55c0111d5d in ev_invoke_pending /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/libev-4.33/ev.c:3770
#12 0x7f55c8c9d774 in kudu::rpc::ReactorThread::InvokePendingCb(ev_loop*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/reactor.cc:205:3
#13 0x7f55c0112ca5 in ev_run /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/libev-4.33/ev.c:4190
#14 0x7f55c8cad370 in ev::loop_ref::run(int) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/uninstrumented/include/ev++.h:211:7
#15 0x7f55c8ca57e5 in kudu::rpc::ReactorThread::RunThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/reactor.cc:510:9
#16 0x7f55c8caa90b in kudu::rpc::ReactorThread::Init()::$_0::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/reactor.cc:197:48
#17 0x7f55c8caa788 in std::_Function_handler<void (), kudu::rpc::ReactorThread::Init()::$_0>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#18 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#19 0x7f55c228e3f8 in kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3
#20 0x7f55c50fe6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Indirect leak of 517 byte(s) in 1 object(s) allocated from:
#0 0x83b998 in operator new[](unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:102
#1 0x7f55c20c1d55 in kudu::faststring::GrowArray(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/faststring.cc:39:38
#2 0x7f55d7a91b38 in kudu::faststring::reserve(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/faststring.h:143:5
#3 0x7f55d7a91a09 in kudu::faststring::resize(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/faststring.h:102:7
#4 0x7f55c8d02a3a in kudu::rpc::serialization::SerializeMessage(google::protobuf::MessageLite const&, kudu::faststring*, int, bool) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/serialization.cc:78:14
#5 0x7f55c8c80d2a in kudu::rpc::RequestPayload::PopulateRequestPayload(google::protobuf::Message const&, std::vector<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> >, std::allocator<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> > > >&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:186:3
#6 0x7f55c8c805ab in kudu::rpc::RequestPayload::CreateRequestPayload(kudu::rpc::RemoteMethod const&, google::protobuf::Message const&, std::vector<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> >, std::allocator<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> > > >&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:163:12
#7 0x7f55c8c94a3f in kudu::rpc::Proxy::AsyncRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:209:22
#8 0x7f55cb8469c9 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/consensus/consensus.proxy.cc:56:3
#9 0x7f55ccc0ceb0 in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:669:21
#10 0x7f55ccc065ed in kudu::consensus::Peer::SendNextRequest(bool, kudu::consensus::MultiRaftConsensusData*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:377:13
#11 0x7f55ccc10ae1 in kudu::consensus::Peer::SignalRequest(bool)::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:213:12
#12 0x7f55ccc107cd in std::_Function_handler<void (), kudu::consensus::Peer::SignalRequest(bool)::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#13 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#14 0x7f55c22af20f in kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7
#15 0x7f55c22b252b in kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48
#16 0x7f55c22b23a8 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_1>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#17 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#18 0x7f55c228e3f8 in kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3
#19 0x7f55c50fe6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Indirect leak of 40 byte(s) in 1 object(s) allocated from:
#0 0x83b838 in operator new(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:99
#1 0x7f55c8373c87 in google::protobuf::Arena::InternalHelper<kudu::rpc::RemoteMethodPB>::New() /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/uninstrumented/include/google/protobuf/arena.h:492:14
#2 0x7f55c8373c87 in kudu::rpc::RemoteMethodPB* google::protobuf::Arena::CreateMessageInternal<kudu::rpc::RemoteMethodPB>(google::protobuf::Arena*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/uninstrumented/include/google/protobuf/arena.h:584:14
#3 0x7f55c8363e00 in kudu::rpc::RemoteMethodPB* google::protobuf::Arena::CreateMaybeMessage<kudu::rpc::RemoteMethodPB>(google::protobuf::Arena*) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/rpc/rpc_header.pb.cc:4322:10
#4 0x7f55c8c89050 in kudu::rpc::RemoteMethodPB* google::protobuf::MessageLite::CreateMaybeMessage<kudu::rpc::RemoteMethodPB>(google::protobuf::Arena*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/uninstrumented/include/google/protobuf/message_lite.h:425:12
#5 0x7f55c8c88ecb in kudu::rpc::RequestHeader::_internal_mutable_remote_method() /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/rpc/rpc_header.pb.h:4908:15
#6 0x7f55c8c872e4 in kudu::rpc::RequestHeader::mutable_remote_method() /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/rpc/rpc_header.pb.h:4914:39
#7 0x7f55c8c804c2 in kudu::rpc::RequestPayload::RequestPayload(kudu::rpc::RemoteMethod const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:155:30
#8 0x7f55c8c86fa7 in std::_MakeUniq<kudu::rpc::RequestPayload>::__single_object std::make_unique<kudu::rpc::RequestPayload, kudu::rpc::RemoteMethod const&>(kudu::rpc::RemoteMethod const&) ../../../include/c++/7.5.0/bits/unique_ptr.h:821:34
#9 0x7f55c8c80580 in kudu::rpc::RequestPayload::CreateRequestPayload(kudu::rpc::RemoteMethod const&, google::protobuf::Message const&, std::vector<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> >, std::allocator<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> > > >&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:162:18
#10 0x7f55c8c94a3f in kudu::rpc::Proxy::AsyncRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:209:22
#11 0x7f55cb8469c9 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/consensus/consensus.proxy.cc:56:3
#12 0x7f55ccc0ceb0 in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:669:21
#13 0x7f55ccc065ed in kudu::consensus::Peer::SendNextRequest(bool, kudu::consensus::MultiRaftConsensusData*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:377:13
#14 0x7f55ccc10ae1 in kudu::consensus::Peer::SignalRequest(bool)::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:213:12
#15 0x7f55ccc107cd in std::_Function_handler<void (), kudu::consensus::Peer::SignalRequest(bool)::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#16 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#17 0x7f55c22af20f in kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7
#18 0x7f55c22b252b in kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48
#19 0x7f55c22b23a8 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_1>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#20 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#21 0x7f55c228e3f8 in kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3
#22 0x7f55c50fe6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Indirect leak of 32 byte(s) in 1 object(s) allocated from:
#0 0x83b838 in operator new(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:99
#1 0xac041f in __gnu_cxx::new_allocator<char>::allocate(unsigned long, void const*) ../../../include/c++/7.5.0/ext/new_allocator.h:111:27
#2 0xac0392 in std::allocator_traits<std::allocator<char> >::allocate(std::allocator<char>&, unsigned long) ../../../include/c++/7.5.0/bits/alloc_traits.h:436:20
#3 0xac0262 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_create(unsigned long&, unsigned long) ../../../include/c++/7.5.0/bits/basic_string.tcc:153:14
#4 0x7f55bf08d87d in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) (.isra.35) /usr/include/c++/7/bits/basic_string.tcc:219
#5 0x7f55bf08da22 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct_aux<char const*>(char const*, char const*, std::__false_type) /usr/include/c++/7/bits/basic_string.h:236
#6 0x7f55bf08da22 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char const*>(char const*, char const*) /usr/include/c++/7/bits/basic_string.h:255
#7 0x7f55bf08da22 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::basic_string(char const*, unsigned long, std::allocator<char> const&) /usr/include/c++/7/bits/basic_string.h:502
#8 0x7f55bf08da22 in CreateString /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/protobuf-3.21.9/src/google/protobuf/arenastring.cc:102
#9 0x7f55bf08da22 in google::protobuf::internal::ArenaStringPtr::Set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Arena*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/protobuf-3.21.9/src/google/protobuf/arenastring.cc:124
#10 0x7f55c8cb7a4f in void kudu::rpc::RemoteMethodPB::set_service_name<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/rpc/rpc_header.pb.h:4544:23
#11 0x7f55c8cb7a4f in kudu::rpc::RemoteMethod::ToPB(kudu::rpc::RemoteMethodPB*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/remote_method.cc:44:7
#12 0x7f55c8c804ce in kudu::rpc::RequestPayload::RequestPayload(kudu::rpc::RemoteMethod const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:155:17
#13 0x7f55c8c86fa7 in std::_MakeUniq<kudu::rpc::RequestPayload>::__single_object std::make_unique<kudu::rpc::RequestPayload, kudu::rpc::RemoteMethod const&>(kudu::rpc::RemoteMethod const&) ../../../include/c++/7.5.0/bits/unique_ptr.h:821:34
#14 0x7f55c8c80580 in kudu::rpc::RequestPayload::CreateRequestPayload(kudu::rpc::RemoteMethod const&, google::protobuf::Message const&, std::vector<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> >, std::allocator<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> > > >&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:162:18
#15 0x7f55c8c94a3f in kudu::rpc::Proxy::AsyncRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:209:22
#16 0x7f55cb8469c9 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/consensus/consensus.proxy.cc:56:3
#17 0x7f55ccc0ceb0 in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:669:21
#18 0x7f55ccc065ed in kudu::consensus::Peer::SendNextRequest(bool, kudu::consensus::MultiRaftConsensusData*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:377:13
#19 0x7f55ccc10ae1 in kudu::consensus::Peer::SignalRequest(bool)::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:213:12
#20 0x7f55ccc107cd in std::_Function_handler<void (), kudu::consensus::Peer::SignalRequest(bool)::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#21 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#22 0x7f55c22af20f in kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7
#23 0x7f55c22b252b in kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48
#24 0x7f55c22b23a8 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_1>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#25 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#26 0x7f55c228e3f8 in kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3
#27 0x7f55c50fe6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Indirect leak of 32 byte(s) in 1 object(s) allocated from:
#0 0x83b838 in operator new(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:99
#1 0x7f55bf08da09 in CreateString /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/protobuf-3.21.9/src/google/protobuf/arenastring.cc:102
#2 0x7f55bf08da09 in google::protobuf::internal::ArenaStringPtr::Set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Arena*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/protobuf-3.21.9/src/google/protobuf/arenastring.cc:124
#3 0x7f55c8cb7b85 in void kudu::rpc::RemoteMethodPB::set_method_name<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/rpc/rpc_header.pb.h:4612:22
#4 0x7f55c8cb7b85 in kudu::rpc::RemoteMethod::ToPB(kudu::rpc::RemoteMethodPB*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/remote_method.cc:45:7
#5 0x7f55c8c804ce in kudu::rpc::RequestPayload::RequestPayload(kudu::rpc::RemoteMethod const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:155:17
#6 0x7f55c8c86fa7 in std::_MakeUniq<kudu::rpc::RequestPayload>::__single_object std::make_unique<kudu::rpc::RequestPayload, kudu::rpc::RemoteMethod const&>(kudu::rpc::RemoteMethod const&) ../../../include/c++/7.5.0/bits/unique_ptr.h:821:34
#7 0x7f55c8c80580 in kudu::rpc::RequestPayload::CreateRequestPayload(kudu::rpc::RemoteMethod const&, google::protobuf::Message const&, std::vector<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> >, std::allocator<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> > > >&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:162:18
#8 0x7f55c8c94a3f in kudu::rpc::Proxy::AsyncRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:209:22
#9 0x7f55cb8469c9 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/consensus/consensus.proxy.cc:56:3
#10 0x7f55ccc0ceb0 in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:669:21
#11 0x7f55ccc065ed in kudu::consensus::Peer::SendNextRequest(bool, kudu::consensus::MultiRaftConsensusData*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:377:13
#12 0x7f55ccc10ae1 in kudu::consensus::Peer::SignalRequest(bool)::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:213:12
#13 0x7f55ccc107cd in std::_Function_handler<void (), kudu::consensus::Peer::SignalRequest(bool)::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#14 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#15 0x7f55c22af20f in kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7
#16 0x7f55c22b252b in kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48
#17 0x7f55c22b23a8 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_1>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#18 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#19 0x7f55c228e3f8 in kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3
#20 0x7f55c50fe6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
Indirect leak of 32 byte(s) in 1 object(s) allocated from:
#0 0x83b838 in operator new(unsigned long) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/asan/asan_new_delete.cpp:99
#1 0x7f55bf08da09 in CreateString /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/protobuf-3.21.9/src/google/protobuf/arenastring.cc:102
#2 0x7f55bf08da09 in google::protobuf::internal::ArenaStringPtr::Set(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Arena*) /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/protobuf-3.21.9/src/google/protobuf/arenastring.cc:124
#3 0x7f55c8cb7a4f in void kudu::rpc::RemoteMethodPB::set_service_name<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/rpc/rpc_header.pb.h:4544:23
#4 0x7f55c8cb7a4f in kudu::rpc::RemoteMethod::ToPB(kudu::rpc::RemoteMethodPB*) const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/remote_method.cc:44:7
#5 0x7f55c8c804ce in kudu::rpc::RequestPayload::RequestPayload(kudu::rpc::RemoteMethod const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:155:17
#6 0x7f55c8c86fa7 in std::_MakeUniq<kudu::rpc::RequestPayload>::__single_object std::make_unique<kudu::rpc::RequestPayload, kudu::rpc::RemoteMethod const&>(kudu::rpc::RemoteMethod const&) ../../../include/c++/7.5.0/bits/unique_ptr.h:821:34
#7 0x7f55c8c80580 in kudu::rpc::RequestPayload::CreateRequestPayload(kudu::rpc::RemoteMethod const&, google::protobuf::Message const&, std::vector<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> >, std::allocator<std::unique_ptr<kudu::rpc::RpcSidecar, std::default_delete<kudu::rpc::RpcSidecar> > > >&&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/outbound_call.cc:162:18
#8 0x7f55c8c94a3f in kudu::rpc::Proxy::AsyncRequest(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, google::protobuf::Message const&, google::protobuf::Message*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/rpc/proxy.cc:209:22
#9 0x7f55cb8469c9 in kudu::consensus::ConsensusServiceProxy::UpdateConsensusAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/build/asan/src/kudu/consensus/consensus.proxy.cc:56:3
#10 0x7f55ccc0ceb0 in kudu::consensus::RpcPeerProxy::UpdateAsync(kudu::consensus::ConsensusRequestPB const&, kudu::consensus::ConsensusResponsePB*, kudu::rpc::RpcController*, std::function<void ()> const&) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:669:21
#11 0x7f55ccc065ed in kudu::consensus::Peer::SendNextRequest(bool, kudu::consensus::MultiRaftConsensusData*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:377:13
#12 0x7f55ccc10ae1 in kudu::consensus::Peer::SignalRequest(bool)::$_2::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/consensus/consensus_peers.cc:213:12
#13 0x7f55ccc107cd in std::_Function_handler<void (), kudu::consensus::Peer::SignalRequest(bool)::$_2>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#14 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#15 0x7f55c22af20f in kudu::ThreadPool::DispatchThread() /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:733:7
#16 0x7f55c22b252b in kudu::ThreadPool::CreateThread()::$_1::operator()() const /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/threadpool.cc:815:48
#17 0x7f55c22b23a8 in std::_Function_handler<void (), kudu::ThreadPool::CreateThread()::$_1>::_M_invoke(std::_Any_data const&) ../../../include/c++/7.5.0/bits/std_function.h:316:2
#18 0xc0c00c in std::function<void ()>::operator()() const ../../../include/c++/7.5.0/bits/std_function.h:706:14
#19 0x7f55c228e3f8 in kudu::Thread::SuperviseThread(void*) /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/thread.cc:694:3
#20 0x7f55c50fe6da in start_thread (/lib/x86_64-linux-gnu/libpthread.so.0+0x76da)
SUMMARY: AddressSanitizer: 901 byte(s) leaked in 7 allocation(s).