[==========] Running 8 tests from 1 test suite.
[----------] Global test environment set-up.
[----------] 8 tests from StopTablets/StopTabletITest
[ RUN ] StopTablets/StopTabletITest.TestSingleStoppedTabletsDontScan/0
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251213 08:57:04.924763 18086 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.17.169.190:45473
I20251213 08:57:04.926367 18086 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251213 08:57:04.927376 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:04.944422 18095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:04.946574 18086 server_base.cc:1047] running on GCE node
W20251213 08:57:04.944643 18092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:04.944651 18093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:06.165076 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:06.165422 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:06.165644 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616226165617 us; error 0 us; skew 500 ppm
I20251213 08:57:06.166661 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:06.175179 18086 webserver.cc:492] Webserver started at http://127.17.169.190:34371/ using document root <none> and password file <none>
I20251213 08:57:06.176318 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:06.176571 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:06.177132 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:06.184165 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/master-0-root/instance:
uuid: "e59ff7e8db004b4b8b55dd75f4c135e7"
format_stamp: "Formatted at 2025-12-13 08:57:06 on dist-test-slave-s4vp"
I20251213 08:57:06.193890 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.009s user 0.010s sys 0.000s
I20251213 08:57:06.200534 18101 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:06.202198 18086 fs_manager.cc:730] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20251213 08:57:06.202663 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/master-0-root
uuid: "e59ff7e8db004b4b8b55dd75f4c135e7"
format_stamp: "Formatted at 2025-12-13 08:57:06 on dist-test-slave-s4vp"
I20251213 08:57:06.203080 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:06.261721 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:06.263736 18086 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251213 08:57:06.264281 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:06.353694 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.190:45473
I20251213 08:57:06.353744 18152 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.190:45473 every 8 connection(s)
I20251213 08:57:06.363672 18153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:06.388684 18153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7: Bootstrap starting.
I20251213 08:57:06.395848 18153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:06.398017 18153 log.cc:826] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7: Log is configured to *not* fsync() on all Append() calls
I20251213 08:57:06.403935 18153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7: No bootstrap required, opened a new log
I20251213 08:57:06.426977 18153 raft_consensus.cc:359] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59ff7e8db004b4b8b55dd75f4c135e7" member_type: VOTER }
I20251213 08:57:06.427665 18153 raft_consensus.cc:385] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:06.427896 18153 raft_consensus.cc:740] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e59ff7e8db004b4b8b55dd75f4c135e7, State: Initialized, Role: FOLLOWER
I20251213 08:57:06.428831 18153 consensus_queue.cc:260] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59ff7e8db004b4b8b55dd75f4c135e7" member_type: VOTER }
I20251213 08:57:06.429361 18153 raft_consensus.cc:399] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:57:06.429617 18153 raft_consensus.cc:493] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:57:06.429935 18153 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:06.434653 18153 raft_consensus.cc:515] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59ff7e8db004b4b8b55dd75f4c135e7" member_type: VOTER }
I20251213 08:57:06.435516 18153 leader_election.cc:304] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e59ff7e8db004b4b8b55dd75f4c135e7; no voters:
I20251213 08:57:06.437523 18153 leader_election.cc:290] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:57:06.437947 18156 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:57:06.440594 18156 raft_consensus.cc:697] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 1 LEADER]: Becoming Leader. State: Replica: e59ff7e8db004b4b8b55dd75f4c135e7, State: Running, Role: LEADER
I20251213 08:57:06.441798 18156 consensus_queue.cc:237] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59ff7e8db004b4b8b55dd75f4c135e7" member_type: VOTER }
I20251213 08:57:06.443215 18153 sys_catalog.cc:565] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [sys.catalog]: configured and running, proceeding with master startup.
I20251213 08:57:06.458463 18158 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e59ff7e8db004b4b8b55dd75f4c135e7. Latest consensus state: current_term: 1 leader_uuid: "e59ff7e8db004b4b8b55dd75f4c135e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59ff7e8db004b4b8b55dd75f4c135e7" member_type: VOTER } }
I20251213 08:57:06.459640 18158 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [sys.catalog]: This master's current role is: LEADER
I20251213 08:57:06.463994 18157 sys_catalog.cc:455] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e59ff7e8db004b4b8b55dd75f4c135e7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e59ff7e8db004b4b8b55dd75f4c135e7" member_type: VOTER } }
I20251213 08:57:06.464710 18157 sys_catalog.cc:458] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [sys.catalog]: This master's current role is: LEADER
I20251213 08:57:06.467336 18163 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251213 08:57:06.484490 18163 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251213 08:57:06.508571 18086 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251213 08:57:06.511332 18163 catalog_manager.cc:1357] Generated new cluster ID: 68a6c1b3563b40bca4e6333fcaca27fb
I20251213 08:57:06.511654 18163 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251213 08:57:06.540787 18163 catalog_manager.cc:1380] Generated new certificate authority record
I20251213 08:57:06.542874 18163 catalog_manager.cc:1514] Loading token signing keys...
I20251213 08:57:06.563866 18163 catalog_manager.cc:6027] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7: Generated new TSK 0
I20251213 08:57:06.565449 18163 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251213 08:57:06.576443 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:06.587893 18175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:06.589834 18176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:06.593858 18086 server_base.cc:1047] running on GCE node
W20251213 08:57:06.594573 18178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:06.595788 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:06.596094 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:06.596311 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616226596294 us; error 0 us; skew 500 ppm
I20251213 08:57:06.597178 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:06.602106 18086 webserver.cc:492] Webserver started at http://127.17.169.129:37233/ using document root <none> and password file <none>
I20251213 08:57:06.602981 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:06.603274 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:06.603664 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:06.605520 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/ts-0-root/instance:
uuid: "45d2fc30231d4f91850a78db8fa37d5f"
format_stamp: "Formatted at 2025-12-13 08:57:06 on dist-test-slave-s4vp"
I20251213 08:57:06.613749 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20251213 08:57:06.619824 18183 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:06.620867 18086 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20251213 08:57:06.621279 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/ts-0-root
uuid: "45d2fc30231d4f91850a78db8fa37d5f"
format_stamp: "Formatted at 2025-12-13 08:57:06 on dist-test-slave-s4vp"
I20251213 08:57:06.621727 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_0.1765616224759991-18086-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:06.644021 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:06.646278 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:06.649745 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:57:06.656423 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:57:06.656865 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:06.657244 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:57:06.657487 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:06.778600 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.129:39305
I20251213 08:57:06.778733 18245 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.129:39305 every 8 connection(s)
I20251213 08:57:06.805220 18246 heartbeater.cc:344] Connected to a master server at 127.17.169.190:45473
I20251213 08:57:06.805729 18246 heartbeater.cc:461] Registering TS with master...
I20251213 08:57:06.806892 18246 heartbeater.cc:507] Master 127.17.169.190:45473 requested a full tablet report, sending...
I20251213 08:57:06.810272 18086 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.02624287s
I20251213 08:57:06.810269 18118 ts_manager.cc:194] Registered new tserver with Master: 45d2fc30231d4f91850a78db8fa37d5f (127.17.169.129:39305)
I20251213 08:57:06.813468 18118 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48824
I20251213 08:57:06.854729 18086 test_util.cc:276] Using random seed: -1878857940
I20251213 08:57:06.899673 18118 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48854:
name: "foo"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251213 08:57:06.939716 18211 tablet_service.cc:1505] Processing CreateTablet for tablet e6be1ce975764306a710bf161cd650ab (DEFAULT_TABLE table=foo [id=5539e71263f740bcb2dda6989653ad39]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:57:06.940898 18211 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6be1ce975764306a710bf161cd650ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:06.954501 18264 tablet_bootstrap.cc:492] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f: Bootstrap starting.
I20251213 08:57:06.959928 18264 tablet_bootstrap.cc:654] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:06.964911 18264 tablet_bootstrap.cc:492] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f: No bootstrap required, opened a new log
I20251213 08:57:06.965293 18264 ts_tablet_manager.cc:1403] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.007s
I20251213 08:57:06.967410 18264 raft_consensus.cc:359] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d2fc30231d4f91850a78db8fa37d5f" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39305 } }
I20251213 08:57:06.967829 18264 raft_consensus.cc:385] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:06.968129 18264 raft_consensus.cc:740] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d2fc30231d4f91850a78db8fa37d5f, State: Initialized, Role: FOLLOWER
I20251213 08:57:06.969949 18264 consensus_queue.cc:260] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d2fc30231d4f91850a78db8fa37d5f" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39305 } }
I20251213 08:57:06.970571 18266 raft_consensus.cc:493] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251213 08:57:06.971051 18266 raft_consensus.cc:515] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d2fc30231d4f91850a78db8fa37d5f" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39305 } }
I20251213 08:57:06.970541 18264 raft_consensus.cc:399] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:57:06.971757 18264 raft_consensus.cc:493] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:57:06.971625 18266 leader_election.cc:304] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45d2fc30231d4f91850a78db8fa37d5f; no voters:
I20251213 08:57:06.972118 18264 raft_consensus.cc:3060] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:06.973541 18266 leader_election.cc:290] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers
I20251213 08:57:06.977005 18264 raft_consensus.cc:515] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d2fc30231d4f91850a78db8fa37d5f" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39305 } }
I20251213 08:57:06.977627 18267 raft_consensus.cc:2764] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20251213 08:57:06.977597 18264 leader_election.cc:304] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45d2fc30231d4f91850a78db8fa37d5f; no voters:
I20251213 08:57:06.978317 18264 leader_election.cc:290] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:57:06.978469 18266 raft_consensus.cc:2804] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:57:06.982252 18266 raft_consensus.cc:697] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 1 LEADER]: Becoming Leader. State: Replica: 45d2fc30231d4f91850a78db8fa37d5f, State: Running, Role: LEADER
I20251213 08:57:06.983104 18246 heartbeater.cc:499] Master 127.17.169.190:45473 was elected leader, sending a full tablet report...
I20251213 08:57:06.983214 18266 consensus_queue.cc:237] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d2fc30231d4f91850a78db8fa37d5f" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39305 } }
I20251213 08:57:06.996896 18118 catalog_manager.cc:5654] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f reported cstate change: term changed from 0 to 1, leader changed from <none> to 45d2fc30231d4f91850a78db8fa37d5f (127.17.169.129). New cstate: current_term: 1 leader_uuid: "45d2fc30231d4f91850a78db8fa37d5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45d2fc30231d4f91850a78db8fa37d5f" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39305 } health_report { overall_health: HEALTHY } } }
I20251213 08:57:07.003880 18264 ts_tablet_manager.cc:1434] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f: Time spent starting tablet: real 0.038s user 0.029s sys 0.009s
I20251213 08:57:07.097031 18086 stop_tablet-itest.cc:169] 0 / 10000 rows inserted
I20251213 08:57:07.525283 18107 maintenance_manager.cc:419] P e59ff7e8db004b4b8b55dd75f4c135e7: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.003577
I20251213 08:57:07.596060 18106 maintenance_manager.cc:643] P e59ff7e8db004b4b8b55dd75f4c135e7: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.070s user 0.047s sys 0.019s Metrics: {"bytes_written":6502,"cfile_init":1,"dirs.queue_time_us":7015,"dirs.run_cpu_time_us":342,"dirs.run_wall_time_us":8996,"drs_written":1,"lbm_read_time_us":344,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2896,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5,"thread_start_us":7279,"threads_started":1}
I20251213 08:57:07.599489 18107 maintenance_manager.cc:419] P e59ff7e8db004b4b8b55dd75f4c135e7: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 419 bytes on disk
I20251213 08:57:07.601636 18106 maintenance_manager.cc:643] P e59ff7e8db004b4b8b55dd75f4c135e7: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":343,"lbm_reads_lt_1ms":4}
I20251213 08:57:08.038623 18247 maintenance_manager.cc:419] P 45d2fc30231d4f91850a78db8fa37d5f: Scheduling FlushMRSOp(e6be1ce975764306a710bf161cd650ab): perf score=0.486106
I20251213 08:57:08.101861 18086 stop_tablet-itest.cc:169] 2800 / 10000 rows inserted
I20251213 08:57:09.052899 18188 maintenance_manager.cc:643] P 45d2fc30231d4f91850a78db8fa37d5f: FlushMRSOp(e6be1ce975764306a710bf161cd650ab) complete. Timing: real 1.013s user 0.786s sys 0.047s Metrics: {"bytes_written":76976,"cfile_init":1,"dirs.queue_time_us":6824,"dirs.run_cpu_time_us":283,"dirs.run_wall_time_us":5135,"drs_written":1,"lbm_read_time_us":263,"lbm_reads_lt_1ms":4,"lbm_write_time_us":7227,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":2650,"spinlock_wait_cycles":296832,"thread_start_us":7198,"threads_started":1}
I20251213 08:57:09.055851 18247 maintenance_manager.cc:419] P 45d2fc30231d4f91850a78db8fa37d5f: Scheduling UndoDeltaBlockGCOp(e6be1ce975764306a710bf161cd650ab): 23749 bytes on disk
I20251213 08:57:09.059262 18188 maintenance_manager.cc:643] P 45d2fc30231d4f91850a78db8fa37d5f: UndoDeltaBlockGCOp(e6be1ce975764306a710bf161cd650ab) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":278,"lbm_reads_lt_1ms":4}
I20251213 08:57:09.111235 18086 stop_tablet-itest.cc:169] 5450 / 10000 rows inserted
I20251213 08:57:10.068872 18247 maintenance_manager.cc:419] P 45d2fc30231d4f91850a78db8fa37d5f: Scheduling FlushMRSOp(e6be1ce975764306a710bf161cd650ab): perf score=0.972434
I20251213 08:57:10.114421 18086 stop_tablet-itest.cc:169] 8850 / 10000 rows inserted
I20251213 08:57:10.645607 18086 stop_tablet-itest.cc:98] Stopping T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f
W20251213 08:57:10.669772 18200 tablet_service.cc:3074] error setting up scanner 7b5caba979f24bfd938861b07d767e08 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "e6be1ce975764306a710bf161cd650ab" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\222\327\364\311\006\"3\n\005slave\022*\n 5539e71263f740bcb2dda6989653ad39\020\001\030\001 \001(\001" signature: "\021\345\t\230\241\022V\335\203\235f\354\214X<L\336\225\2077\303e\257\313\207\340QY\341\334\244\313Zw\245\254\254\352\2439\367\241x\014\215{\tCs\206G\336\225*7\275%\034\016\224\302>\001\264""\021\345\t\230\241\022V\335\203\235f\354\214X<L\336\225\2077\303e\257\313\207\340QY\341\334\244\313Zw\245\254\254\352\2439\367\241x\014\215{\tCs\206G\336\225*7\275%\034\016\224\302>\001\264" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4356ecdb6aa49cda11bcbb5309953b1"
W20251213 08:57:10.879156 18200 tablet_service.cc:3074] error setting up scanner 37e3846062844a188b2026aa85bf602a with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "e6be1ce975764306a710bf161cd650ab" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\222\327\364\311\006\"3\n\005slave\022*\n 5539e71263f740bcb2dda6989653ad39\020\001\030\001 \001(\001" signature: "\021\345\t\230\241\022V\335\203\235f\354\214X<L\336\225\2077\303e\257\313\207\340QY\341\334\244\313Zw\245\254\254\352\2439\367\241x\014\215{\tCs\206G\336\225*7\275%\034\016\224\302>\001\264""\021\345\t\230\241\022V\335\203\235f\354\214X<L\336\225\2077\303e\257\313\207\340QY\341\334\244\313Zw\245\254\254\352\2439\367\241x\014\215{\tCs\206G\336\225*7\275%\034\016\224\302>\001\264" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4356ecdb6aa49cda11bcbb5309953b1"
W20251213 08:57:11.129848 18188 tablet_replica_mm_ops.cc:214] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f: failed to flush MRS: Aborted: MVCC is closed
I20251213 08:57:11.130146 18188 maintenance_manager.cc:643] P 45d2fc30231d4f91850a78db8fa37d5f: FlushMRSOp(e6be1ce975764306a710bf161cd650ab) complete. Timing: real 1.061s user 0.943s sys 0.004s Metrics: {"cfile_init":1,"dirs.queue_time_us":866,"dirs.run_cpu_time_us":335,"dirs.run_wall_time_us":1767,"lbm_read_time_us":166,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3479,"lbm_writes_lt_1ms":28,"spinlock_wait_cycles":205184,"thread_start_us":583,"threads_started":1}
I20251213 08:57:11.286260 18086 scanner-internal.cc:172] Scan of tablet e6be1ce975764306a710bf161cd650ab at 45d2fc30231d4f91850a78db8fa37d5f (127.17.169.129:39305) deadline exceeded after 0.500s
W20251213 08:57:11.286801 18086 scanner-internal.cc:458] Time spent opening tablet: real 0.639s user 0.006s sys 0.001s
I20251213 08:57:11.286993 18086 stop_tablet-itest.cc:199] Scanner opened with status: Timed out: exceeded configured scan timeout of 0.500s: after 3 scan attempts: Scan RPC to 127.17.169.129:39305 timed out after -0.138s (SENT): Aborted: MVCC is closed
W20251213 08:57:11.289274 18200 tablet_service.cc:3074] error setting up scanner e3ba2c37eb254cb597f4e47721e4c0e7 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "e6be1ce975764306a710bf161cd650ab" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\222\327\364\311\006\"3\n\005slave\022*\n 5539e71263f740bcb2dda6989653ad39\020\001\030\001 \001(\001" signature: "\021\345\t\230\241\022V\335\203\235f\354\214X<L\336\225\2077\303e\257\313\207\340QY\341\334\244\313Zw\245\254\254\352\2439\367\241x\014\215{\tCs\206G\336\225*7\275%\034\016\224\302>\001\264""\021\345\t\230\241\022V\335\203\235f\354\214X<L\336\225\2077\303e\257\313\207\340QY\341\334\244\313Zw\245\254\254\352\2439\367\241x\014\215{\tCs\206G\336\225*7\275%\034\016\224\302>\001\264" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "d4356ecdb6aa49cda11bcbb5309953b1"
I20251213 08:57:11.290459 18086 tablet_server.cc:178] TabletServer@127.17.169.129:0 shutting down...
W20251213 08:57:11.294137 18187 connection.cc:441] server connection from 127.0.0.1:57640 torn down before Call kudu.tserver.TabletServerService.Scan from 127.0.0.1:57640 (request call id 2) could send its response
I20251213 08:57:11.327858 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:57:11.328552 18086 tablet_replica.cc:333] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f: stopping tablet replica
I20251213 08:57:11.329120 18086 raft_consensus.cc:2243] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:57:11.329638 18086 raft_consensus.cc:2272] T e6be1ce975764306a710bf161cd650ab P 45d2fc30231d4f91850a78db8fa37d5f [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:57:11.500552 18086 tablet_server.cc:195] TabletServer@127.17.169.129:0 shutdown complete.
I20251213 08:57:11.512023 18086 master.cc:561] Master@127.17.169.190:45473 shutting down...
I20251213 08:57:11.611788 18086 raft_consensus.cc:2243] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:57:11.612713 18086 raft_consensus.cc:2272] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:57:11.613107 18086 tablet_replica.cc:333] T 00000000000000000000000000000000 P e59ff7e8db004b4b8b55dd75f4c135e7: stopping tablet replica
I20251213 08:57:11.633797 18086 master.cc:583] Master@127.17.169.190:45473 shutdown complete.
[ OK ] StopTablets/StopTabletITest.TestSingleStoppedTabletsDontScan/0 (6736 ms)
[ RUN ] StopTablets/StopTabletITest.TestSingleStoppedTabletsDontScan/1
I20251213 08:57:11.659217 18086 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.17.169.190:43191
I20251213 08:57:11.660255 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:11.666141 18293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:11.666349 18294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:11.669963 18086 server_base.cc:1047] running on GCE node
W20251213 08:57:11.669920 18297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:11.671305 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:11.671554 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:11.671721 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616231671707 us; error 0 us; skew 500 ppm
I20251213 08:57:11.672338 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:11.675407 18086 webserver.cc:492] Webserver started at http://127.17.169.190:40617/ using document root <none> and password file <none>
I20251213 08:57:11.676033 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:11.676254 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:11.676548 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:11.677981 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/master-0-root/instance:
uuid: "9dbcf55b28a242ceb91ed54b16714290"
format_stamp: "Formatted at 2025-12-13 08:57:11 on dist-test-slave-s4vp"
I20251213 08:57:11.684041 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20251213 08:57:11.688555 18301 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:11.689594 18086 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20251213 08:57:11.690079 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/master-0-root
uuid: "9dbcf55b28a242ceb91ed54b16714290"
format_stamp: "Formatted at 2025-12-13 08:57:11 on dist-test-slave-s4vp"
I20251213 08:57:11.690546 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:11.714859 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:11.716156 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:11.767246 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.190:43191
I20251213 08:57:11.767333 18352 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.190:43191 every 8 connection(s)
I20251213 08:57:11.772378 18353 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:11.781692 18353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290: Bootstrap starting.
I20251213 08:57:11.785979 18353 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:11.790645 18353 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290: No bootstrap required, opened a new log
I20251213 08:57:11.792644 18353 raft_consensus.cc:359] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbcf55b28a242ceb91ed54b16714290" member_type: VOTER }
I20251213 08:57:11.793108 18353 raft_consensus.cc:385] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:11.793419 18353 raft_consensus.cc:740] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dbcf55b28a242ceb91ed54b16714290, State: Initialized, Role: FOLLOWER
I20251213 08:57:11.794119 18353 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbcf55b28a242ceb91ed54b16714290" member_type: VOTER }
I20251213 08:57:11.794791 18353 raft_consensus.cc:399] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:57:11.795104 18353 raft_consensus.cc:493] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:57:11.795451 18353 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:11.884639 18353 raft_consensus.cc:515] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbcf55b28a242ceb91ed54b16714290" member_type: VOTER }
I20251213 08:57:11.885468 18353 leader_election.cc:304] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9dbcf55b28a242ceb91ed54b16714290; no voters:
I20251213 08:57:11.887022 18353 leader_election.cc:290] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:57:11.887281 18356 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:57:11.891077 18353 sys_catalog.cc:565] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [sys.catalog]: configured and running, proceeding with master startup.
I20251213 08:57:11.891532 18356 raft_consensus.cc:697] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 1 LEADER]: Becoming Leader. State: Replica: 9dbcf55b28a242ceb91ed54b16714290, State: Running, Role: LEADER
I20251213 08:57:11.892196 18356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbcf55b28a242ceb91ed54b16714290" member_type: VOTER }
I20251213 08:57:11.897156 18357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9dbcf55b28a242ceb91ed54b16714290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbcf55b28a242ceb91ed54b16714290" member_type: VOTER } }
I20251213 08:57:11.897378 18358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9dbcf55b28a242ceb91ed54b16714290. Latest consensus state: current_term: 1 leader_uuid: "9dbcf55b28a242ceb91ed54b16714290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9dbcf55b28a242ceb91ed54b16714290" member_type: VOTER } }
I20251213 08:57:11.897677 18357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [sys.catalog]: This master's current role is: LEADER
I20251213 08:57:11.897956 18358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [sys.catalog]: This master's current role is: LEADER
I20251213 08:57:11.900339 18365 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251213 08:57:11.905788 18365 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251213 08:57:11.909689 18086 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251213 08:57:11.915261 18365 catalog_manager.cc:1357] Generated new cluster ID: 91b5ec85c02d49e299c9bc684a99e98a
I20251213 08:57:11.915573 18365 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251213 08:57:11.943868 18365 catalog_manager.cc:1380] Generated new certificate authority record
I20251213 08:57:11.945363 18365 catalog_manager.cc:1514] Loading token signing keys...
I20251213 08:57:11.958698 18365 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290: Generated new TSK 0
I20251213 08:57:11.959501 18365 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251213 08:57:11.976774 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:11.982955 18374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:11.984405 18375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:11.986264 18086 server_base.cc:1047] running on GCE node
W20251213 08:57:11.986819 18377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:11.987877 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:11.988155 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:11.988320 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616231988297 us; error 0 us; skew 500 ppm
I20251213 08:57:11.988938 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:11.994632 18086 webserver.cc:492] Webserver started at http://127.17.169.129:45797/ using document root <none> and password file <none>
I20251213 08:57:11.995270 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:11.995491 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:11.995800 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:11.997134 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/ts-0-root/instance:
uuid: "41ff6b217de54914baf6d04745777fc2"
format_stamp: "Formatted at 2025-12-13 08:57:11 on dist-test-slave-s4vp"
I20251213 08:57:12.002408 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.007s
I20251213 08:57:12.006578 18382 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:12.007468 18086 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20251213 08:57:12.007754 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/ts-0-root
uuid: "41ff6b217de54914baf6d04745777fc2"
format_stamp: "Formatted at 2025-12-13 08:57:11 on dist-test-slave-s4vp"
I20251213 08:57:12.008024 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestSingleStoppedTabletsDontScan_1.1765616224759991-18086-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:12.046720 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:12.047883 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:12.049469 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:57:12.052035 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:57:12.052254 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:12.052534 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:57:12.052682 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:12.097771 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.129:41281
I20251213 08:57:12.097899 18444 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.129:41281 every 8 connection(s)
I20251213 08:57:12.117197 18445 heartbeater.cc:344] Connected to a master server at 127.17.169.190:43191
I20251213 08:57:12.117671 18445 heartbeater.cc:461] Registering TS with master...
I20251213 08:57:12.118546 18445 heartbeater.cc:507] Master 127.17.169.190:43191 requested a full tablet report, sending...
I20251213 08:57:12.120779 18318 ts_manager.cc:194] Registered new tserver with Master: 41ff6b217de54914baf6d04745777fc2 (127.17.169.129:41281)
I20251213 08:57:12.121711 18086 internal_mini_cluster.cc:371] 1 TS(s) registered with all masters after 0.020088715s
I20251213 08:57:12.123171 18318 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37830
I20251213 08:57:12.157243 18086 test_util.cc:276] Using random seed: -1873555426
I20251213 08:57:12.181286 18318 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37854:
name: "foo"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 1
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251213 08:57:12.218631 18410 tablet_service.cc:1505] Processing CreateTablet for tablet 29615089fbf04ba0bc9466216c12861b (DEFAULT_TABLE table=foo [id=5c16436543da42c8906a01cca91bb6a6]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:57:12.219789 18410 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29615089fbf04ba0bc9466216c12861b. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:12.234583 18463 tablet_bootstrap.cc:492] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2: Bootstrap starting.
I20251213 08:57:12.240259 18463 tablet_bootstrap.cc:654] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:12.245399 18463 tablet_bootstrap.cc:492] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2: No bootstrap required, opened a new log
I20251213 08:57:12.245867 18463 ts_tablet_manager.cc:1403] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.005s
I20251213 08:57:12.248029 18463 raft_consensus.cc:359] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41ff6b217de54914baf6d04745777fc2" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41281 } }
I20251213 08:57:12.248510 18463 raft_consensus.cc:385] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:12.248771 18463 raft_consensus.cc:740] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41ff6b217de54914baf6d04745777fc2, State: Initialized, Role: FOLLOWER
I20251213 08:57:12.249361 18463 consensus_queue.cc:260] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41ff6b217de54914baf6d04745777fc2" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41281 } }
I20251213 08:57:12.249830 18463 raft_consensus.cc:399] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:57:12.250077 18463 raft_consensus.cc:493] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:57:12.250396 18463 raft_consensus.cc:3060] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:12.284883 18463 raft_consensus.cc:515] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41ff6b217de54914baf6d04745777fc2" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41281 } }
I20251213 08:57:12.285630 18463 leader_election.cc:304] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41ff6b217de54914baf6d04745777fc2; no voters:
I20251213 08:57:12.287189 18463 leader_election.cc:290] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:57:12.287565 18465 raft_consensus.cc:2804] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:57:12.289233 18445 heartbeater.cc:499] Master 127.17.169.190:43191 was elected leader, sending a full tablet report...
I20251213 08:57:12.289942 18463 ts_tablet_manager.cc:1434] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2: Time spent starting tablet: real 0.044s user 0.012s sys 0.005s
I20251213 08:57:12.291067 18465 raft_consensus.cc:697] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 1 LEADER]: Becoming Leader. State: Replica: 41ff6b217de54914baf6d04745777fc2, State: Running, Role: LEADER
I20251213 08:57:12.291749 18465 consensus_queue.cc:237] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41ff6b217de54914baf6d04745777fc2" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41281 } }
I20251213 08:57:12.299870 18318 catalog_manager.cc:5654] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41ff6b217de54914baf6d04745777fc2 (127.17.169.129). New cstate: current_term: 1 leader_uuid: "41ff6b217de54914baf6d04745777fc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41ff6b217de54914baf6d04745777fc2" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41281 } health_report { overall_health: HEALTHY } } }
I20251213 08:57:12.376338 18086 stop_tablet-itest.cc:169] 0 / 10000 rows inserted
I20251213 08:57:12.975570 18307 maintenance_manager.cc:419] P 9dbcf55b28a242ceb91ed54b16714290: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.003577
I20251213 08:57:13.108035 18306 maintenance_manager.cc:643] P 9dbcf55b28a242ceb91ed54b16714290: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.132s user 0.048s sys 0.020s Metrics: {"bytes_written":6504,"cfile_init":1,"dirs.queue_time_us":18864,"dirs.run_cpu_time_us":316,"dirs.run_wall_time_us":79251,"drs_written":1,"lbm_read_time_us":311,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3070,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5,"spinlock_wait_cycles":27904,"thread_start_us":18531,"threads_started":1}
I20251213 08:57:13.111011 18307 maintenance_manager.cc:419] P 9dbcf55b28a242ceb91ed54b16714290: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 394 bytes on disk
I20251213 08:57:13.113571 18306 maintenance_manager.cc:643] P 9dbcf55b28a242ceb91ed54b16714290: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":391,"lbm_reads_lt_1ms":4}
I20251213 08:57:13.357702 18446 maintenance_manager.cc:419] P 41ff6b217de54914baf6d04745777fc2: Scheduling FlushMRSOp(29615089fbf04ba0bc9466216c12861b): perf score=0.486106
I20251213 08:57:13.383147 18086 stop_tablet-itest.cc:169] 3700 / 10000 rows inserted
I20251213 08:57:14.398741 18086 stop_tablet-itest.cc:169] 7150 / 10000 rows inserted
I20251213 08:57:14.443367 18387 maintenance_manager.cc:643] P 41ff6b217de54914baf6d04745777fc2: FlushMRSOp(29615089fbf04ba0bc9466216c12861b) complete. Timing: real 1.085s user 0.943s sys 0.056s Metrics: {"bytes_written":108078,"cfile_init":1,"dirs.queue_time_us":4234,"dirs.run_cpu_time_us":383,"dirs.run_wall_time_us":82980,"drs_written":1,"lbm_read_time_us":262,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3409,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":3700,"thread_start_us":3963,"threads_started":1}
I20251213 08:57:14.446300 18446 maintenance_manager.cc:419] P 41ff6b217de54914baf6d04745777fc2: Scheduling UndoDeltaBlockGCOp(29615089fbf04ba0bc9466216c12861b): 33239 bytes on disk
I20251213 08:57:14.447983 18387 maintenance_manager.cc:643] P 41ff6b217de54914baf6d04745777fc2: UndoDeltaBlockGCOp(29615089fbf04ba0bc9466216c12861b) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":288,"lbm_reads_lt_1ms":4}
I20251213 08:57:15.157531 18086 stop_tablet-itest.cc:98] Stopping T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2
W20251213 08:57:15.175470 18398 tablet_service.cc:3074] error setting up scanner 8988264230ae45a9bae6e772398943d1 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "29615089fbf04ba0bc9466216c12861b" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\227\327\364\311\006\"3\n\005slave\022*\n 5c16436543da42c8906a01cca91bb6a6\020\001\030\001 \001(\001" signature: "Z\235\022\231\310\'w\"\236\225}7\263\342\3364\353 p\371=\336l=\267M\232\371\031\302\251$\032#GL\021\3410@ 3n\311\230\336-V\272\206hIY\346{0+\275\033\207\366\242\240\216""Z\235\022\231\310\'w\"\236\225}7\263\342\3364\353 p\371=\336l=\267M\232\371\031\302\251$\032#GL\021\3410@ 3n\311\230\336-V\272\206hIY\346{0+\275\033\207\366\242\240\216" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "b3d1abce50624c6a83a024fdd9b75697"
W20251213 08:57:15.385174 18398 tablet_service.cc:3074] error setting up scanner bebc5b70586a4f808039ecbc631400df with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "29615089fbf04ba0bc9466216c12861b" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\227\327\364\311\006\"3\n\005slave\022*\n 5c16436543da42c8906a01cca91bb6a6\020\001\030\001 \001(\001" signature: "Z\235\022\231\310\'w\"\236\225}7\263\342\3364\353 p\371=\336l=\267M\232\371\031\302\251$\032#GL\021\3410@ 3n\311\230\336-V\272\206hIY\346{0+\275\033\207\366\242\240\216""Z\235\022\231\310\'w\"\236\225}7\263\342\3364\353 p\371=\336l=\267M\232\371\031\302\251$\032#GL\021\3410@ 3n\311\230\336-V\272\206hIY\346{0+\275\033\207\366\242\240\216" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "b3d1abce50624c6a83a024fdd9b75697"
I20251213 08:57:15.791749 18086 scanner-internal.cc:172] Scan of tablet 29615089fbf04ba0bc9466216c12861b at 41ff6b217de54914baf6d04745777fc2 (127.17.169.129:41281) deadline exceeded after 0.500s
W20251213 08:57:15.792289 18086 scanner-internal.cc:458] Time spent opening tablet: real 0.634s user 0.006s sys 0.000s
I20251213 08:57:15.792562 18086 stop_tablet-itest.cc:199] Scanner opened with status: Timed out: exceeded configured scan timeout of 0.500s: after 3 scan attempts: Scan RPC to 127.17.169.129:41281 timed out after -0.132s (SENT): Aborted: MVCC is closed
W20251213 08:57:15.793951 18398 tablet_service.cc:3074] error setting up scanner 2ecfd47a360042d39c625c8bdb84df23 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "29615089fbf04ba0bc9466216c12861b" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\227\327\364\311\006\"3\n\005slave\022*\n 5c16436543da42c8906a01cca91bb6a6\020\001\030\001 \001(\001" signature: "Z\235\022\231\310\'w\"\236\225}7\263\342\3364\353 p\371=\336l=\267M\232\371\031\302\251$\032#GL\021\3410@ 3n\311\230\336-V\272\206hIY\346{0+\275\033\207\366\242\240\216""Z\235\022\231\310\'w\"\236\225}7\263\342\3364\353 p\371=\336l=\267M\232\371\031\302\251$\032#GL\021\3410@ 3n\311\230\336-V\272\206hIY\346{0+\275\033\207\366\242\240\216" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "b3d1abce50624c6a83a024fdd9b75697"
I20251213 08:57:15.795876 18086 tablet_server.cc:178] TabletServer@127.17.169.129:0 shutting down...
W20251213 08:57:15.795779 18383 connection.cc:441] server connection from 127.0.0.1:36792 torn down before Call kudu.tserver.TabletServerService.Scan from 127.0.0.1:36792 (request call id 2) could send its response
I20251213 08:57:15.833129 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:57:15.834617 18086 tablet_replica.cc:333] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2: stopping tablet replica
I20251213 08:57:15.835121 18086 raft_consensus.cc:2243] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:57:15.835685 18086 raft_consensus.cc:2272] T 29615089fbf04ba0bc9466216c12861b P 41ff6b217de54914baf6d04745777fc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:57:15.866429 18086 tablet_server.cc:195] TabletServer@127.17.169.129:0 shutdown complete.
I20251213 08:57:15.877976 18086 master.cc:561] Master@127.17.169.190:43191 shutting down...
I20251213 08:57:15.897267 18086 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:57:15.897799 18086 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:57:15.898083 18086 tablet_replica.cc:333] T 00000000000000000000000000000000 P 9dbcf55b28a242ceb91ed54b16714290: stopping tablet replica
I20251213 08:57:15.915966 18086 master.cc:583] Master@127.17.169.190:43191 shutdown complete.
[ OK ] StopTablets/StopTabletITest.TestSingleStoppedTabletsDontScan/1 (4287 ms)
[ RUN ] StopTablets/StopTabletITest.TestStoppedTabletsScansGetRedirected/0
I20251213 08:57:15.946962 18086 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.17.169.190:38027
I20251213 08:57:15.948098 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:15.953673 18492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:15.953536 18491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:15.954710 18494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:15.955753 18086 server_base.cc:1047] running on GCE node
I20251213 08:57:15.956414 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:15.956619 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:15.956764 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616235956748 us; error 0 us; skew 500 ppm
I20251213 08:57:15.957238 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:15.959649 18086 webserver.cc:492] Webserver started at http://127.17.169.190:33393/ using document root <none> and password file <none>
I20251213 08:57:15.960130 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:15.960330 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:15.960580 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:15.961678 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/master-0-root/instance:
uuid: "8df39f3a2a8346d79c6fb8ca9e662354"
format_stamp: "Formatted at 2025-12-13 08:57:15 on dist-test-slave-s4vp"
I20251213 08:57:15.965957 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20251213 08:57:15.969197 18499 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:15.969873 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251213 08:57:15.970142 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/master-0-root
uuid: "8df39f3a2a8346d79c6fb8ca9e662354"
format_stamp: "Formatted at 2025-12-13 08:57:15 on dist-test-slave-s4vp"
I20251213 08:57:15.970458 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:15.983605 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:15.984783 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:16.024755 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.190:38027
I20251213 08:57:16.024812 18550 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.190:38027 every 8 connection(s)
I20251213 08:57:16.028540 18551 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:16.039183 18551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354: Bootstrap starting.
I20251213 08:57:16.043833 18551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:16.048671 18551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354: No bootstrap required, opened a new log
I20251213 08:57:16.050768 18551 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df39f3a2a8346d79c6fb8ca9e662354" member_type: VOTER }
I20251213 08:57:16.051167 18551 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:16.051400 18551 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8df39f3a2a8346d79c6fb8ca9e662354, State: Initialized, Role: FOLLOWER
I20251213 08:57:16.051945 18551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df39f3a2a8346d79c6fb8ca9e662354" member_type: VOTER }
I20251213 08:57:16.052358 18551 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:57:16.052577 18551 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:57:16.052843 18551 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:16.057336 18551 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df39f3a2a8346d79c6fb8ca9e662354" member_type: VOTER }
I20251213 08:57:16.057842 18551 leader_election.cc:304] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8df39f3a2a8346d79c6fb8ca9e662354; no voters:
I20251213 08:57:16.058935 18551 leader_election.cc:290] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:57:16.059351 18554 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:57:16.060655 18554 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 1 LEADER]: Becoming Leader. State: Replica: 8df39f3a2a8346d79c6fb8ca9e662354, State: Running, Role: LEADER
I20251213 08:57:16.061255 18554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df39f3a2a8346d79c6fb8ca9e662354" member_type: VOTER }
I20251213 08:57:16.061874 18551 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [sys.catalog]: configured and running, proceeding with master startup.
I20251213 08:57:16.067114 18556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8df39f3a2a8346d79c6fb8ca9e662354. Latest consensus state: current_term: 1 leader_uuid: "8df39f3a2a8346d79c6fb8ca9e662354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df39f3a2a8346d79c6fb8ca9e662354" member_type: VOTER } }
I20251213 08:57:16.067426 18555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8df39f3a2a8346d79c6fb8ca9e662354" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8df39f3a2a8346d79c6fb8ca9e662354" member_type: VOTER } }
I20251213 08:57:16.067821 18556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [sys.catalog]: This master's current role is: LEADER
I20251213 08:57:16.068146 18555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [sys.catalog]: This master's current role is: LEADER
I20251213 08:57:16.070503 18561 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251213 08:57:16.075944 18561 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251213 08:57:16.078395 18086 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251213 08:57:16.084672 18561 catalog_manager.cc:1357] Generated new cluster ID: 700501750de947619d406466c019764c
I20251213 08:57:16.084970 18561 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251213 08:57:16.096061 18561 catalog_manager.cc:1380] Generated new certificate authority record
I20251213 08:57:16.097270 18561 catalog_manager.cc:1514] Loading token signing keys...
I20251213 08:57:16.107321 18561 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354: Generated new TSK 0
I20251213 08:57:16.107858 18561 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251213 08:57:16.114993 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:16.120756 18572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:16.121946 18573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:16.125036 18086 server_base.cc:1047] running on GCE node
W20251213 08:57:16.124970 18575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:16.126134 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:16.126386 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:16.126550 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616236126527 us; error 0 us; skew 500 ppm
I20251213 08:57:16.127033 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:16.129356 18086 webserver.cc:492] Webserver started at http://127.17.169.129:34177/ using document root <none> and password file <none>
I20251213 08:57:16.129824 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:16.130002 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:16.130311 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:16.131556 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-0-root/instance:
uuid: "a074fef9a84040058bd02b7afcc60da6"
format_stamp: "Formatted at 2025-12-13 08:57:16 on dist-test-slave-s4vp"
I20251213 08:57:16.135854 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20251213 08:57:16.138980 18580 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:16.139716 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251213 08:57:16.139993 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-0-root
uuid: "a074fef9a84040058bd02b7afcc60da6"
format_stamp: "Formatted at 2025-12-13 08:57:16 on dist-test-slave-s4vp"
I20251213 08:57:16.140277 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:16.155874 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:16.156982 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:16.158418 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:57:16.160609 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:57:16.160806 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:16.161027 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:57:16.161192 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:16.201022 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.129:39489
I20251213 08:57:16.201157 18642 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.129:39489 every 8 connection(s)
I20251213 08:57:16.205785 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:16.213377 18647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:16.216351 18648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:16.217942 18650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:16.218350 18086 server_base.cc:1047] running on GCE node
I20251213 08:57:16.218720 18643 heartbeater.cc:344] Connected to a master server at 127.17.169.190:38027
I20251213 08:57:16.219085 18643 heartbeater.cc:461] Registering TS with master...
I20251213 08:57:16.219470 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:16.219688 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:16.219758 18643 heartbeater.cc:507] Master 127.17.169.190:38027 requested a full tablet report, sending...
I20251213 08:57:16.219869 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616236219850 us; error 0 us; skew 500 ppm
I20251213 08:57:16.220412 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:16.222015 18516 ts_manager.cc:194] Registered new tserver with Master: a074fef9a84040058bd02b7afcc60da6 (127.17.169.129:39489)
I20251213 08:57:16.223102 18086 webserver.cc:492] Webserver started at http://127.17.169.130:40719/ using document root <none> and password file <none>
I20251213 08:57:16.223604 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:16.223793 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:16.224031 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:16.223984 18516 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50456
I20251213 08:57:16.225400 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-1-root/instance:
uuid: "04d79b9a9172401eb02e16f67cfa050b"
format_stamp: "Formatted at 2025-12-13 08:57:16 on dist-test-slave-s4vp"
I20251213 08:57:16.229710 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20251213 08:57:16.233122 18655 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:16.234006 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251213 08:57:16.234315 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-1-root
uuid: "04d79b9a9172401eb02e16f67cfa050b"
format_stamp: "Formatted at 2025-12-13 08:57:16 on dist-test-slave-s4vp"
I20251213 08:57:16.234601 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:16.250424 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:16.251592 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:16.252980 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:57:16.255242 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:57:16.255517 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:16.255751 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:57:16.255904 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:16.318578 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.130:34353
I20251213 08:57:16.318687 18717 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.130:34353 every 8 connection(s)
I20251213 08:57:16.324896 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:16.333498 18721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:16.336099 18722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:16.337311 18718 heartbeater.cc:344] Connected to a master server at 127.17.169.190:38027
I20251213 08:57:16.337759 18718 heartbeater.cc:461] Registering TS with master...
I20251213 08:57:16.338750 18718 heartbeater.cc:507] Master 127.17.169.190:38027 requested a full tablet report, sending...
I20251213 08:57:16.338932 18086 server_base.cc:1047] running on GCE node
W20251213 08:57:16.339078 18724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:16.340189 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:16.340412 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:16.340596 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616236340576 us; error 0 us; skew 500 ppm
I20251213 08:57:16.340881 18516 ts_manager.cc:194] Registered new tserver with Master: 04d79b9a9172401eb02e16f67cfa050b (127.17.169.130:34353)
I20251213 08:57:16.341127 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:16.342501 18516 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50458
I20251213 08:57:16.344607 18086 webserver.cc:492] Webserver started at http://127.17.169.131:40241/ using document root <none> and password file <none>
I20251213 08:57:16.345114 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:16.345279 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:16.345551 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:16.346853 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-2-root/instance:
uuid: "8ba557a87917421da986e5cdc07833ad"
format_stamp: "Formatted at 2025-12-13 08:57:16 on dist-test-slave-s4vp"
I20251213 08:57:16.351234 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20251213 08:57:16.354393 18729 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:16.355154 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20251213 08:57:16.355441 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-2-root
uuid: "8ba557a87917421da986e5cdc07833ad"
format_stamp: "Formatted at 2025-12-13 08:57:16 on dist-test-slave-s4vp"
I20251213 08:57:16.355721 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_0.1765616224759991-18086-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:16.381337 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:16.382563 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:16.384531 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:57:16.387215 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:57:16.387411 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:16.387636 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:57:16.387789 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:16.460152 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.131:44225
I20251213 08:57:16.460269 18791 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.131:44225 every 8 connection(s)
I20251213 08:57:16.474484 18792 heartbeater.cc:344] Connected to a master server at 127.17.169.190:38027
I20251213 08:57:16.474895 18792 heartbeater.cc:461] Registering TS with master...
I20251213 08:57:16.475554 18792 heartbeater.cc:507] Master 127.17.169.190:38027 requested a full tablet report, sending...
I20251213 08:57:16.477408 18516 ts_manager.cc:194] Registered new tserver with Master: 8ba557a87917421da986e5cdc07833ad (127.17.169.131:44225)
I20251213 08:57:16.478364 18086 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013628102s
I20251213 08:57:16.479116 18516 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:50472
I20251213 08:57:16.514117 18086 test_util.cc:276] Using random seed: -1869198546
I20251213 08:57:16.541610 18516 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50512:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251213 08:57:16.544066 18516 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251213 08:57:16.601120 18757 tablet_service.cc:1505] Processing CreateTablet for tablet 9149b1ceecc14d1298b47ea50058dc7c (DEFAULT_TABLE table=test-workload [id=edbfe6fe6b6546b8af7a4c5d7d990453]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:57:16.602134 18608 tablet_service.cc:1505] Processing CreateTablet for tablet 9149b1ceecc14d1298b47ea50058dc7c (DEFAULT_TABLE table=test-workload [id=edbfe6fe6b6546b8af7a4c5d7d990453]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:57:16.602591 18757 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9149b1ceecc14d1298b47ea50058dc7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:16.603432 18608 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9149b1ceecc14d1298b47ea50058dc7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:16.603724 18683 tablet_service.cc:1505] Processing CreateTablet for tablet 9149b1ceecc14d1298b47ea50058dc7c (DEFAULT_TABLE table=test-workload [id=edbfe6fe6b6546b8af7a4c5d7d990453]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:57:16.604956 18683 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9149b1ceecc14d1298b47ea50058dc7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:16.619714 18813 tablet_bootstrap.cc:492] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6: Bootstrap starting.
I20251213 08:57:16.621555 18814 tablet_bootstrap.cc:492] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b: Bootstrap starting.
I20251213 08:57:16.624814 18815 tablet_bootstrap.cc:492] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad: Bootstrap starting.
I20251213 08:57:16.628801 18813 tablet_bootstrap.cc:654] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:16.631235 18814 tablet_bootstrap.cc:654] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:16.631449 18815 tablet_bootstrap.cc:654] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:16.639138 18814 tablet_bootstrap.cc:492] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b: No bootstrap required, opened a new log
I20251213 08:57:16.639225 18813 tablet_bootstrap.cc:492] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6: No bootstrap required, opened a new log
I20251213 08:57:16.639231 18815 tablet_bootstrap.cc:492] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad: No bootstrap required, opened a new log
I20251213 08:57:16.639716 18814 ts_tablet_manager.cc:1403] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b: Time spent bootstrapping tablet: real 0.019s user 0.016s sys 0.000s
I20251213 08:57:16.639789 18813 ts_tablet_manager.cc:1403] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6: Time spent bootstrapping tablet: real 0.020s user 0.017s sys 0.000s
I20251213 08:57:16.639850 18815 ts_tablet_manager.cc:1403] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20251213 08:57:16.642344 18815 raft_consensus.cc:359] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } }
I20251213 08:57:16.642613 18813 raft_consensus.cc:359] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } }
I20251213 08:57:16.643042 18815 raft_consensus.cc:385] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:16.642792 18814 raft_consensus.cc:359] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } }
I20251213 08:57:16.643342 18813 raft_consensus.cc:385] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:16.643461 18814 raft_consensus.cc:385] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:16.643383 18815 raft_consensus.cc:740] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba557a87917421da986e5cdc07833ad, State: Initialized, Role: FOLLOWER
I20251213 08:57:16.643803 18813 raft_consensus.cc:740] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a074fef9a84040058bd02b7afcc60da6, State: Initialized, Role: FOLLOWER
I20251213 08:57:16.644434 18815 consensus_queue.cc:260] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } }
I20251213 08:57:16.644568 18813 consensus_queue.cc:260] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } }
I20251213 08:57:16.646564 18814 raft_consensus.cc:740] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d79b9a9172401eb02e16f67cfa050b, State: Initialized, Role: FOLLOWER
I20251213 08:57:16.647128 18643 heartbeater.cc:499] Master 127.17.169.190:38027 was elected leader, sending a full tablet report...
I20251213 08:57:16.647403 18813 ts_tablet_manager.cc:1434] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20251213 08:57:16.647588 18792 heartbeater.cc:499] Master 127.17.169.190:38027 was elected leader, sending a full tablet report...
I20251213 08:57:16.648845 18814 consensus_queue.cc:260] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } }
I20251213 08:57:16.651804 18815 ts_tablet_manager.cc:1434] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad: Time spent starting tablet: real 0.011s user 0.004s sys 0.002s
I20251213 08:57:16.652376 18718 heartbeater.cc:499] Master 127.17.169.190:38027 was elected leader, sending a full tablet report...
I20251213 08:57:16.653610 18814 ts_tablet_manager.cc:1434] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b: Time spent starting tablet: real 0.013s user 0.005s sys 0.002s
W20251213 08:57:16.705405 18644 tablet.cc:2378] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251213 08:57:16.736619 18819 raft_consensus.cc:493] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251213 08:57:16.737054 18819 raft_consensus.cc:515] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } }
I20251213 08:57:16.739253 18819 leader_election.cc:290] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ba557a87917421da986e5cdc07833ad (127.17.169.131:44225), 04d79b9a9172401eb02e16f67cfa050b (127.17.169.130:34353)
I20251213 08:57:16.748565 18767 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" candidate_uuid: "a074fef9a84040058bd02b7afcc60da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba557a87917421da986e5cdc07833ad" is_pre_election: true
I20251213 08:57:16.749310 18767 raft_consensus.cc:2468] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a074fef9a84040058bd02b7afcc60da6 in term 0.
I20251213 08:57:16.750001 18693 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" candidate_uuid: "a074fef9a84040058bd02b7afcc60da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d79b9a9172401eb02e16f67cfa050b" is_pre_election: true
I20251213 08:57:16.750653 18693 raft_consensus.cc:2468] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a074fef9a84040058bd02b7afcc60da6 in term 0.
I20251213 08:57:16.750751 18582 leader_election.cc:304] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ba557a87917421da986e5cdc07833ad, a074fef9a84040058bd02b7afcc60da6; no voters:
I20251213 08:57:16.751658 18819 raft_consensus.cc:2804] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251213 08:57:16.751937 18819 raft_consensus.cc:493] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251213 08:57:16.752187 18819 raft_consensus.cc:3060] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:16.756923 18819 raft_consensus.cc:515] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } }
I20251213 08:57:16.758486 18819 leader_election.cc:290] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [CANDIDATE]: Term 1 election: Requested vote from peers 8ba557a87917421da986e5cdc07833ad (127.17.169.131:44225), 04d79b9a9172401eb02e16f67cfa050b (127.17.169.130:34353)
I20251213 08:57:16.759333 18767 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" candidate_uuid: "a074fef9a84040058bd02b7afcc60da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ba557a87917421da986e5cdc07833ad"
I20251213 08:57:16.759433 18693 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" candidate_uuid: "a074fef9a84040058bd02b7afcc60da6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "04d79b9a9172401eb02e16f67cfa050b"
I20251213 08:57:16.759786 18767 raft_consensus.cc:3060] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:16.759893 18693 raft_consensus.cc:3060] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:16.764459 18767 raft_consensus.cc:2468] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a074fef9a84040058bd02b7afcc60da6 in term 1.
I20251213 08:57:16.764571 18693 raft_consensus.cc:2468] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a074fef9a84040058bd02b7afcc60da6 in term 1.
I20251213 08:57:16.765452 18582 leader_election.cc:304] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ba557a87917421da986e5cdc07833ad, a074fef9a84040058bd02b7afcc60da6; no voters:
I20251213 08:57:16.766201 18819 raft_consensus.cc:2804] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:57:16.767729 18819 raft_consensus.cc:697] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 1 LEADER]: Becoming Leader. State: Replica: a074fef9a84040058bd02b7afcc60da6, State: Running, Role: LEADER
I20251213 08:57:16.768376 18819 consensus_queue.cc:237] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } }
I20251213 08:57:16.777580 18514 catalog_manager.cc:5654] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a074fef9a84040058bd02b7afcc60da6 (127.17.169.129). New cstate: current_term: 1 leader_uuid: "a074fef9a84040058bd02b7afcc60da6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a074fef9a84040058bd02b7afcc60da6" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 39489 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 } health_report { overall_health: UNKNOWN } } }
I20251213 08:57:16.792768 18086 test_util.cc:276] Using random seed: -1868919899
I20251213 08:57:16.815562 18086 test_workload.cc:405] TestWorkload: Skipping table creation because table test-workload already exists
I20251213 08:57:16.873839 18086 stop_tablet-itest.cc:222] 0 / 10000 rows inserted
I20251213 08:57:16.893937 18767 raft_consensus.cc:1275] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [term 1 FOLLOWER]: Refusing update from remote peer a074fef9a84040058bd02b7afcc60da6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:57:16.894281 18693 raft_consensus.cc:1275] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [term 1 FOLLOWER]: Refusing update from remote peer a074fef9a84040058bd02b7afcc60da6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:57:16.895648 18824 consensus_queue.cc:1048] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ba557a87917421da986e5cdc07833ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 44225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251213 08:57:16.896443 18819 consensus_queue.cc:1048] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "04d79b9a9172401eb02e16f67cfa050b" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 34353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251213 08:57:16.925261 18837 mvcc.cc:204] Tried to move back new op lower bound from 7231964106302705664 to 7231964105810460672. Current Snapshot: MvccSnapshot[applied={T|T < 7231964106302705664}]
I20251213 08:57:17.241814 18505 maintenance_manager.cc:419] P 8df39f3a2a8346d79c6fb8ca9e662354: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.003577
I20251213 08:57:17.442451 18504 maintenance_manager.cc:643] P 8df39f3a2a8346d79c6fb8ca9e662354: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.196s user 0.065s sys 0.027s Metrics: {"bytes_written":6635,"cfile_init":1,"dirs.queue_time_us":93085,"dirs.run_cpu_time_us":302,"dirs.run_wall_time_us":3098,"drs_written":1,"lbm_read_time_us":322,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3022,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5,"thread_start_us":45675,"threads_started":1}
I20251213 08:57:17.445415 18505 maintenance_manager.cc:419] P 8df39f3a2a8346d79c6fb8ca9e662354: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 491 bytes on disk
I20251213 08:57:17.447609 18504 maintenance_manager.cc:643] P 8df39f3a2a8346d79c6fb8ca9e662354: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":363,"lbm_reads_lt_1ms":4}
I20251213 08:57:17.709721 18644 maintenance_manager.cc:419] P a074fef9a84040058bd02b7afcc60da6: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.242942
I20251213 08:57:17.719337 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.242942
I20251213 08:57:17.842111 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.242942
I20251213 08:57:17.894593 18086 stop_tablet-itest.cc:222] 1600 / 10000 rows inserted
I20251213 08:57:18.338217 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.618s user 0.384s sys 0.038s Metrics: {"bytes_written":39549,"cfile_init":1,"dirs.queue_time_us":25693,"dirs.run_cpu_time_us":291,"dirs.run_wall_time_us":7464,"drs_written":1,"lbm_read_time_us":257,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2497,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1350,"spinlock_wait_cycles":276352,"thread_start_us":25405,"threads_started":1}
I20251213 08:57:18.341063 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 11795 bytes on disk
I20251213 08:57:18.342900 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":327,"lbm_reads_lt_1ms":4}
I20251213 08:57:18.725205 18585 maintenance_manager.cc:643] P a074fef9a84040058bd02b7afcc60da6: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.015s user 0.395s sys 0.024s Metrics: {"bytes_written":40936,"cfile_init":1,"dirs.queue_time_us":6822,"dirs.run_cpu_time_us":283,"dirs.run_wall_time_us":8820,"drs_written":1,"lbm_read_time_us":256,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2538,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1400,"spinlock_wait_cycles":32204544,"thread_start_us":7037,"threads_started":1}
I20251213 08:57:18.752720 18644 maintenance_manager.cc:419] P a074fef9a84040058bd02b7afcc60da6: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 12217 bytes on disk
I20251213 08:57:18.754417 18585 maintenance_manager.cc:643] P a074fef9a84040058bd02b7afcc60da6: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":297,"lbm_reads_lt_1ms":4}
I20251213 08:57:18.814438 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.971s user 0.426s sys 0.027s Metrics: {"bytes_written":44680,"cfile_init":1,"dirs.queue_time_us":5176,"dirs.run_cpu_time_us":266,"dirs.run_wall_time_us":75371,"drs_written":1,"lbm_read_time_us":291,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2543,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1550,"thread_start_us":5352,"threads_started":1}
I20251213 08:57:18.817332 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 13548 bytes on disk
I20251213 08:57:18.819028 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":279,"lbm_reads_lt_1ms":4}
I20251213 08:57:18.905794 18086 stop_tablet-itest.cc:222] 2850 / 10000 rows inserted
I20251213 08:57:19.349745 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.242942
I20251213 08:57:19.763060 18644 maintenance_manager.cc:419] P a074fef9a84040058bd02b7afcc60da6: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.486106
W20251213 08:57:19.783409 18835 scanner-internal.cc:458] Time spent opening tablet: real 0.607s user 0.002s sys 0.000s
I20251213 08:57:19.827582 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.486106
I20251213 08:57:19.913048 18086 stop_tablet-itest.cc:222] 4400 / 10000 rows inserted
W20251213 08:57:20.704977 18835 scanner-internal.cc:458] Time spent opening tablet: real 0.627s user 0.002s sys 0.000s
I20251213 08:57:20.793326 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.443s user 0.645s sys 0.003s Metrics: {"bytes_written":69413,"cfile_init":1,"dirs.queue_time_us":28103,"dirs.run_cpu_time_us":333,"dirs.run_wall_time_us":3737,"drs_written":1,"lbm_read_time_us":14691,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":3,"lbm_write_time_us":3226,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2350,"spinlock_wait_cycles":250112,"thread_start_us":56304,"threads_started":1}
I20251213 08:57:20.797211 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 20832 bytes on disk
I20251213 08:57:20.799059 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":331,"lbm_reads_lt_1ms":4}
I20251213 08:57:20.801632 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=1.014670
I20251213 08:57:20.921967 18086 stop_tablet-itest.cc:222] 5600 / 10000 rows inserted
I20251213 08:57:21.200346 18585 maintenance_manager.cc:643] P a074fef9a84040058bd02b7afcc60da6: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.436s user 0.753s sys 0.016s Metrics: {"bytes_written":82296,"cfile_init":1,"dirs.queue_time_us":6897,"dirs.run_cpu_time_us":353,"dirs.run_wall_time_us":6267,"drs_written":1,"lbm_read_time_us":288,"lbm_reads_lt_1ms":4,"lbm_write_time_us":25427,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":2850,"thread_start_us":7128,"threads_started":1}
I20251213 08:57:21.212529 18644 maintenance_manager.cc:419] P a074fef9a84040058bd02b7afcc60da6: Scheduling CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=1.014516
I20251213 08:57:21.704162 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.876s user 0.741s sys 0.020s Metrics: {"bytes_written":79613,"cfile_init":1,"dirs.queue_time_us":16816,"dirs.run_cpu_time_us":328,"dirs.run_wall_time_us":1313,"drs_written":1,"lbm_read_time_us":280,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3136,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2750,"spinlock_wait_cycles":229760,"thread_start_us":16526,"threads_started":1}
I20251213 08:57:21.720906 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=1.014535
I20251213 08:57:21.936431 18086 stop_tablet-itest.cc:222] 6700 / 10000 rows inserted
W20251213 08:57:22.665952 18835 scanner-internal.cc:458] Time spent opening tablet: real 0.680s user 0.002s sys 0.000s
I20251213 08:57:22.743103 18585 maintenance_manager.cc:643] P a074fef9a84040058bd02b7afcc60da6: CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.530s user 0.858s sys 0.053s Metrics: {"bytes_written":122427,"cfile_cache_hit":23,"cfile_cache_hit_bytes":146897,"cfile_cache_miss":15,"cfile_cache_miss_bytes":148781,"cfile_init":4,"delta_iterators_relevant":4,"dirs.queue_time_us":28035,"dirs.run_cpu_time_us":303,"dirs.run_wall_time_us":1327,"drs_written":1,"lbm_read_time_us":2899,"lbm_reads_lt_1ms":31,"lbm_write_time_us":3986,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":65566,"rows_written":4250,"thread_start_us":27752,"threads_started":1}
I20251213 08:57:22.746722 18644 maintenance_manager.cc:419] P a074fef9a84040058bd02b7afcc60da6: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 38360 bytes on disk
I20251213 08:57:22.755275 18585 maintenance_manager.cc:643] P a074fef9a84040058bd02b7afcc60da6: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":266,"lbm_reads_lt_1ms":4}
I20251213 08:57:22.757511 18644 maintenance_manager.cc:419] P a074fef9a84040058bd02b7afcc60da6: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.486106
I20251213 08:57:22.880666 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 2.078s user 0.738s sys 0.031s Metrics: {"bytes_written":108116,"cfile_cache_hit":10,"cfile_cache_hit_bytes":67765,"cfile_cache_miss":28,"cfile_cache_miss_bytes":189129,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":98039,"dirs.run_cpu_time_us":331,"dirs.run_wall_time_us":19094,"drs_written":1,"lbm_read_time_us":3991,"lbm_reads_lt_1ms":48,"lbm_write_time_us":4002,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":55581,"rows_written":3700,"thread_start_us":40335,"threads_started":1}
I20251213 08:57:22.916911 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 33338 bytes on disk
I20251213 08:57:22.918681 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":277,"lbm_reads_lt_1ms":4}
I20251213 08:57:22.921518 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.486106
I20251213 08:57:22.940716 18086 stop_tablet-itest.cc:222] 7800 / 10000 rows inserted
W20251213 08:57:23.218897 18833 meta_cache.cc:1261] Time spent looking up entry by key: real 0.089s user 0.004s sys 0.000s
I20251213 08:57:23.285171 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.563s user 0.870s sys 0.024s Metrics: {"bytes_written":123812,"cfile_cache_hit":10,"cfile_cache_hit_bytes":78715,"cfile_cache_miss":28,"cfile_cache_miss_bytes":219873,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":6951,"dirs.run_cpu_time_us":304,"dirs.run_wall_time_us":10569,"drs_written":1,"lbm_read_time_us":38160,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":51,"lbm_write_time_us":3829,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":64531,"rows_written":4300,"thread_start_us":7159,"threads_started":1}
I20251213 08:57:23.288408 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.486106
I20251213 08:57:23.944281 18086 stop_tablet-itest.cc:222] 8800 / 10000 rows inserted
W20251213 08:57:24.591761 18835 scanner-internal.cc:458] Time spent opening tablet: real 0.661s user 0.001s sys 0.000s
I20251213 08:57:24.951177 18086 stop_tablet-itest.cc:222] 10000 / 10000 rows inserted
I20251213 08:57:25.064682 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 2.142s user 1.090s sys 0.035s Metrics: {"bytes_written":119731,"cfile_init":1,"dirs.queue_time_us":6816,"dirs.run_cpu_time_us":294,"dirs.run_wall_time_us":24456,"drs_written":1,"lbm_read_time_us":269,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3737,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":4150,"spinlock_wait_cycles":16768384,"thread_start_us":7012,"threads_started":1}
I20251213 08:57:25.109952 18086 stop_tablet-itest.cc:98] Stopping T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6
I20251213 08:57:25.127270 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 37381 bytes on disk
I20251213 08:57:25.129092 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":290,"lbm_reads_lt_1ms":4}
I20251213 08:57:25.131711 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=1.014741
W20251213 08:57:25.234272 18883 mvcc.cc:118] aborting op with timestamp 7231964140082774016 in state 0; MVCC is closed
W20251213 08:57:25.236012 18883 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:40424: Illegal state: Not Applying op; the tablet is stopped
W20251213 08:57:25.239702 18585 tablet_replica_mm_ops.cc:214] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6: failed to flush MRS: Aborted: Failed to flush new tablet metadata: MVCC is closed
I20251213 08:57:25.240258 18585 maintenance_manager.cc:643] P a074fef9a84040058bd02b7afcc60da6: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 2.459s user 0.905s sys 0.007s Metrics: {"cfile_init":1,"dirs.queue_time_us":12085,"dirs.run_cpu_time_us":289,"dirs.run_wall_time_us":9345,"lbm_read_time_us":260,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3672,"lbm_writes_lt_1ms":26,"spinlock_wait_cycles":235264,"thread_start_us":12289,"threads_started":1}
W20251213 08:57:25.301026 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.303860 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.306630 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.363893 18673 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.365254 18670 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.367345 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.368278 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.393800 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.403450 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.407531 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.421980 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.425236 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.429526 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.442471 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.446467 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.462543 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.474758 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.479655 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.486102 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.488888 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.493757 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.504261 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.510053 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.533504 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.540220 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.543427 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.563277 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.565529 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
I20251213 08:57:25.565160 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 2.271s user 0.989s sys 0.036s Metrics: {"bytes_written":111742,"cfile_init":1,"dirs.queue_time_us":66477,"dirs.run_cpu_time_us":393,"dirs.run_wall_time_us":56962,"drs_written":1,"lbm_read_time_us":268,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3849,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":3850,"thread_start_us":97883,"threads_started":1}
W20251213 08:57:25.569149 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
I20251213 08:57:25.571832 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 73456 bytes on disk
I20251213 08:57:25.573915 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":444,"lbm_reads_lt_1ms":8}
I20251213 08:57:25.575897 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=1.014841
W20251213 08:57:25.583823 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.595484 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.603756 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.620712 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.629062 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.644189 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.649858 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.653690 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.657749 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.666648 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.679356 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.705721 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.709517 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.713232 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.725749 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.730751 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.731771 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.738184 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.747490 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.797333 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.801070 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.805351 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.808418 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.828267 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.832340 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.836364 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.840265 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.898701 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.898963 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.902695 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.904079 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.926679 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.927886 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.931263 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.931954 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:25.989213 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.996052 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:25.999593 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.001207 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.018461 18596 tablet_service.cc:3074] error setting up scanner e29f86c525964e7d830a6bc5052e362c with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964140082774016 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "e9b819c4bd6c4ca08523398a70664dae"
W20251213 08:57:26.023257 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.033591 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.037181 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.038089 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.093981 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.103008 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.107331 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.107615 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.127580 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.142120 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.142954 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.187853 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.206015 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.218094 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.225149 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.225593 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.246213 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.258920 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.265129 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.308913 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.328260 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.340556 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.349642 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.350276 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.371055 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.384598 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.391885 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.443192 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.460697 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.483403 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.486660 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.490470 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.505643 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
I20251213 08:57:26.518219 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.386s user 1.254s sys 0.016s Metrics: {"bytes_written":225865,"cfile_cache_hit":10,"cfile_cache_hit_bytes":143579,"cfile_cache_miss":30,"cfile_cache_miss_bytes":402563,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":769,"dirs.run_cpu_time_us":345,"dirs.run_wall_time_us":1333,"drs_written":1,"lbm_read_time_us":27616,"lbm_reads_1-10_ms":2,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":47,"lbm_write_time_us":4774,"lbm_writes_lt_1ms":30,"num_input_rowsets":2,"peak_mem_usage":119571,"rows_written":7850,"thread_start_us":1081,"threads_started":2,"wal-append.queue_time_us":948}
I20251213 08:57:26.520915 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 72780 bytes on disk
I20251213 08:57:26.522264 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":191,"lbm_reads_lt_1ms":4}
I20251213 08:57:26.523814 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.486106
W20251213 08:57:26.535899 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.537681 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.588753 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.612426 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.636515 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.641224 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.660122 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.666404 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.694654 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.715065 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.739156 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.761623 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.794365 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.797487 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.817452 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.821552 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.851665 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.874938 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.910842 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.924803 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:26.963584 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.968307 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
I20251213 08:57:26.972704 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.396s user 1.338s sys 0.026s Metrics: {"bytes_written":233639,"cfile_cache_hit":10,"cfile_cache_hit_bytes":149204,"cfile_cache_miss":30,"cfile_cache_miss_bytes":417826,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":10510,"dirs.run_cpu_time_us":1297,"dirs.run_wall_time_us":9209,"drs_written":1,"lbm_read_time_us":9669,"lbm_reads_1-10_ms":3,"lbm_reads_lt_1ms":47,"lbm_write_time_us":4740,"lbm_writes_lt_1ms":30,"mutex_wait_us":14,"num_input_rowsets":2,"peak_mem_usage":124146,"rows_written":8150,"thread_start_us":9983,"threads_started":4,"wal-append.queue_time_us":921}
I20251213 08:57:26.975862 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=0.242942
W20251213 08:57:26.977690 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:26.989663 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.026496 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.044536 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
I20251213 08:57:27.070441 18744 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Scan from 127.0.0.1:54642 (request call id 37) took 1043 ms. Trace:
I20251213 08:57:27.070765 18744 rpcz_store.cc:276] 1213 08:57:26.027498 (+ 0us) service_pool.cc:168] Inserting onto call queue
1213 08:57:26.027642 (+ 144us) service_pool.cc:225] Handling call
1213 08:57:26.028275 (+ 633us) tablet_service.cc:2884] Created scanner 62a19fe5538647d8969ed30b4d80915c for tablet 9149b1ceecc14d1298b47ea50058dc7c, query id is e9b819c4bd6c4ca08523398a70664dae
1213 08:57:26.029452 (+ 1177us) tablet_service.cc:3024] Creating iterator
1213 08:57:26.029540 (+ 88us) tablet_service.cc:3402] Waiting safe time to advance
1213 08:57:26.430375 (+400835us) tablet_service.cc:3409] Waiting for operations to commit
1213 08:57:26.430510 (+ 135us) tablet_service.cc:3425] All operations in snapshot committed. Waited for 400880 microseconds
1213 08:57:26.430852 (+ 342us) tablet_service.cc:3049] Iterator created
1213 08:57:26.562549 (+131697us) tablet_service.cc:3071] Iterator init: OK
1213 08:57:26.562688 (+ 139us) tablet_service.cc:3114] has_more: true
1213 08:57:26.562919 (+ 231us) tablet_service.cc:3131] Continuing scan request
1213 08:57:26.563140 (+ 221us) tablet_service.cc:3195] Found scanner 62a19fe5538647d8969ed30b4d80915c for tablet 9149b1ceecc14d1298b47ea50058dc7c, query id is e9b819c4bd6c4ca08523398a70664dae
1213 08:57:27.069133 (+505993us) tablet_service.cc:3270] Deadline expired after scanning 3584 rows: responding with 146944 bytes (batch size 1048576)
1213 08:57:27.070365 (+ 1232us) inbound_call.cc:173] Queueing success response
Metrics: {"cfile_cache_hit":8,"cfile_cache_hit_bytes":143464,"cfile_cache_miss":6,"cfile_cache_miss_bytes":59084,"lbm_read_time_us":851,"lbm_reads_lt_1ms":6,"rowset_iterators":2,"scanner_bytes_read":59060}
W20251213 08:57:27.075497 18835 scanner-internal.cc:458] Time spent opening tablet: real 1.061s user 0.003s sys 0.000s
I20251213 08:57:27.075392 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.551s user 0.533s sys 0.004s Metrics: {"bytes_written":73159,"cfile_init":1,"dirs.queue_time_us":905,"dirs.run_cpu_time_us":254,"dirs.run_wall_time_us":901,"drs_written":1,"lbm_read_time_us":166,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2665,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2500,"thread_start_us":628,"threads_started":1}
I20251213 08:57:27.078701 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=1.014664
W20251213 08:57:27.082110 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.094206 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.141243 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.145125 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.160821 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.157908 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.204331 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.226547 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.260821 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.289379 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.327778 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.332123 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.343914 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.349848 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.397034 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.402456 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.470027 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.474645 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.529682 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.533654 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.538246 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.541162 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
I20251213 08:57:27.579243 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: FlushMRSOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.601s user 0.570s sys 0.016s Metrics: {"bytes_written":65456,"cfile_init":1,"dirs.queue_time_us":119,"dirs.run_cpu_time_us":334,"dirs.run_wall_time_us":867,"drs_written":1,"lbm_read_time_us":295,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3093,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":2200,"spinlock_wait_cycles":8064}
I20251213 08:57:27.582564 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 94877 bytes on disk
I20251213 08:57:27.585225 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":695,"lbm_reads_lt_1ms":8}
I20251213 08:57:27.587841 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c): perf score=1.014656
W20251213 08:57:27.600000 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.610659 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.613445 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.669871 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.732625 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.735494 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.740211 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.748862 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.816247 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.818833 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.872956 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.878629 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:27.942737 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.946110 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.960843 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:27.964874 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.029461 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.035107 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.087082 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.096683 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.155550 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.170957 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.175240 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.182860 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.198807 18596 tablet_service.cc:3074] error setting up scanner 421ecfb7d9e548ce88915f191c0a9569 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964147999703040 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "6353c3b97e6a498f95f21b9d871567a1"
W20251213 08:57:28.246116 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.256281 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.299268 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.324950 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.372289 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.394977 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.397037 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.407007 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.473302 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.482782 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.526672 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.556505 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.602815 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.635749 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.643767 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.646543 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.713897 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.732787 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.761915 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.818457 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.843623 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
I20251213 08:57:28.852152 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.773s user 1.739s sys 0.028s Metrics: {"bytes_written":300394,"cfile_cache_hit":16,"cfile_cache_hit_bytes":269763,"cfile_cache_miss":28,"cfile_cache_miss_bytes":459155,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":872,"dirs.run_cpu_time_us":321,"dirs.run_wall_time_us":1163,"drs_written":1,"lbm_read_time_us":7221,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":51,"lbm_write_time_us":5688,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":32,"num_input_rowsets":2,"peak_mem_usage":157687,"rows_written":10350,"thread_start_us":1045,"threads_started":2,"wal-append.queue_time_us":815}
I20251213 08:57:28.855393 18793 maintenance_manager.cc:419] P 8ba557a87917421da986e5cdc07833ad: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 96916 bytes on disk
I20251213 08:57:28.857064 18734 maintenance_manager.cc:643] P 8ba557a87917421da986e5cdc07833ad: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":263,"lbm_reads_lt_1ms":4}
W20251213 08:57:28.881098 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.896696 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:28.901022 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
I20251213 08:57:28.931953 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: CompactRowSetsOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 1.344s user 1.291s sys 0.046s Metrics: {"bytes_written":300394,"cfile_cache_hit":10,"cfile_cache_hit_bytes":193228,"cfile_cache_miss":32,"cfile_cache_miss_bytes":535482,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":3831,"dirs.run_cpu_time_us":1388,"dirs.run_wall_time_us":5315,"drs_written":1,"lbm_read_time_us":7615,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":51,"lbm_write_time_us":4035,"lbm_writes_lt_1ms":33,"mutex_wait_us":1166,"num_input_rowsets":2,"peak_mem_usage":157782,"rows_written":10350,"thread_start_us":3105,"threads_started":5,"wal-append.queue_time_us":880}
I20251213 08:57:28.934270 18719 maintenance_manager.cc:419] P 04d79b9a9172401eb02e16f67cfa050b: Scheduling UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c): 96916 bytes on disk
I20251213 08:57:28.935694 18660 maintenance_manager.cc:643] P 04d79b9a9172401eb02e16f67cfa050b: UndoDeltaBlockGCOp(9149b1ceecc14d1298b47ea50058dc7c) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":266,"lbm_reads_lt_1ms":4}
W20251213 08:57:28.959359 18835 scanner-internal.cc:458] Time spent opening tablet: real 0.764s user 0.003s sys 0.000s
W20251213 08:57:28.962729 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:28.976532 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.012171 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.061952 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.094676 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.125268 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.138630 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.142949 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.208415 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.219020 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.266798 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.313102 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.333210 18596 tablet_service.cc:3074] error setting up scanner 1118c0aaf1f44530b9d9e4a0b411f7a0 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964155717337088 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "50c6c42a95be4e68845dbdbaa2f61e98"
W20251213 08:57:29.353370 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.379418 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.392088 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.399783 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.467150 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.477712 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.530546 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.579803 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.617415 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.646407 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.658342 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.668866 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.737584 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.749161 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.796949 18596 tablet_service.cc:3074] error setting up scanner be8b7f839cef48df9f1419f2a7850f6d with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964158482702336 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5e5fa271d43343f2b1cfe594577ab072"
W20251213 08:57:29.799892 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.857260 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:29.892925 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.923591 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.934475 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:29.951536 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.017570 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.028121 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.081297 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.140151 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.174674 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.211826 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.213984 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.236847 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.306566 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.307723 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.369196 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.371857 18596 tablet_service.cc:3074] error setting up scanner 9b547ab403a744c281e7566e1425b68d with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964160829702144 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "6d1e3eb8975d4eeca41c72cd33b0e3a1"
W20251213 08:57:30.434562 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.466061 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.501866 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.506063 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.533958 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.597498 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.601622 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.664206 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.734249 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.737475 18596 tablet_service.cc:3074] error setting up scanner 6daf7811b9074673865ec3a5909502cf with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964162343325696 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "69f9e99a0d77442c8f551edd79d0a4f3"
W20251213 08:57:30.761910 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.796420 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.798707 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.836851 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:30.897068 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.897419 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:30.969689 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.043597 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.073782 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.104825 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.104849 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.149601 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.208143 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.208341 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.283658 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.291147 18595 tablet_service.cc:3074] error setting up scanner cf7eaa0daa084a9aa72cd8bcb5bb1c9b with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964164613873664 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "c6a4e53c8f9c416f9064954a00c3e71b"
W20251213 08:57:31.361222 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.389691 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.423724 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.424132 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.466802 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.530709 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.531816 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.609457 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.688868 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.716584 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.741724 18595 tablet_service.cc:3074] error setting up scanner 235c0033679b41fe87ef76d34ec287b8 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964166450003968 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "b8756215ea564bb390a7352c4524d680"
W20251213 08:57:31.750941 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.751360 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:31.798128 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.859629 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.862819 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:31.942178 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.023976 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.052770 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.086637 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.088765 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.137861 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.199139 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.201385 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.238438 18596 tablet_service.cc:3074] error setting up scanner 2a1cede797994707bb499d2c84464006 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964168486502400 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "a610ce21f9f04eed8ffa80f08c1bc18c"
W20251213 08:57:32.279318 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.369237 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.396342 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.430353 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.430619 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.484479 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.547116 18745 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.549039 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.636515 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.722148 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.756469 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.787437 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.789753 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.841849 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:32.872493 18596 tablet_service.cc:3074] error setting up scanner 0cc06f25dca3497fa25d2d86e601d6d6 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964171078234112 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "7dd07509def74c12948c77fcaedb94c1"
W20251213 08:57:32.908869 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:32.911173 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.000233 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.088335 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.119611 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.152446 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.155453 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.209126 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.272090 18596 tablet_service.cc:3074] error setting up scanner 9232189c7bb747558d2dceee60d71427 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964172701147136 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "9db93b8b088d4689be49ac9380bbe2cb"
W20251213 08:57:33.275326 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.367174 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.402074 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.464174 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.492424 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.526993 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.527057 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.590595 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.651551 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.744071 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.755002 18596 tablet_service.cc:3074] error setting up scanner 8125e60511f24b1aa2a3357e7c675490 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964174695755776 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "3f8c3f6cd25f4fd6b08cf7064710ee7d"
W20251213 08:57:33.781697 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.847949 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:33.870270 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.907578 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.908581 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:33.978747 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:34.038573 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.130235 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.169142 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.199250 18596 tablet_service.cc:3074] error setting up scanner 946fc6e8d6ae40ab80b813dc9055ef08 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964176502358016 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "1dc76c0354ee40fe98f2fc0e4c1f5944"
W20251213 08:57:34.244132 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.263370 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:34.297190 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:34.302433 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:34.377461 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:34.431712 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.530887 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.564149 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.647035 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.665510 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:34.698626 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:34.698626 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:34.758720 18596 tablet_service.cc:3074] error setting up scanner 754ef7e22fef4b5a9f4c934d72c6df41 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964178807799808 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "ab8eea6beecd43aba991dbd5ba95e199"
W20251213 08:57:34.779381 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:34.835830 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.940745 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:34.971333 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.052784 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.077008 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.110388 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.113972 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.191746 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.252995 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.319499 18596 tablet_service.cc:3074] error setting up scanner da584b40b1af4d0fb1f564a6b1400cf0 with request: Aborted: MVCC is closed: new_scan_request { tablet_id: "9149b1ceecc14d1298b47ea50058dc7c" projected_columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } projected_columns { name: "int_val" type: INT32 is_key: false is_nullable: false immutable: false } projected_columns { name: "string_val" type: STRING is_key: false is_nullable: true immutable: false } read_mode: READ_AT_SNAPSHOT propagated_timestamp: 7231964181098217472 cache_blocks: true order_mode: ORDERED row_format_flags: 0 authz_token { token_data: "\010\230\327\364\311\006\"3\n\005slave\022*\n edbfe6fe6b6546b8af7a4c5d7d990453\020\001\030\001 \001(\001" signature: "!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004""!\356z2\032l\350\252\342/<gJ\222\030\260R\266\237\324x\210\350>>\264\300\035\321\206\014\324\334\275\'\235\321\200\344C\273\365\360u\213U\205\320sWL\232!\246\342\314E\020\364\242\277\225\363\004" signing_key_seq_num: 0 } } call_seq_id: 0 query_id: "5d53df7d6be249deb36d106ae17edf67"
W20251213 08:57:35.354307 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.415704 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.473278 18744 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.499186 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.531307 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.556061 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.615427 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.668941 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.786295 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.843079 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.904681 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:35.928274 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.954865 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:35.984452 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.045212 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.099484 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:36.218401 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:36.271524 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:36.337318 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:36.365748 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.393265 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.414948 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.482654 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.537339 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:36.658587 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:36.709028 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:36.778501 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:36.804147 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.830273 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.855752 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.925555 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:36.978883 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:37.103091 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:37.159264 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:37.224763 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:37.254457 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:37.278995 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:37.309875 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:37.376430 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:37.429024 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:37.562564 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:37.618826 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:37.688944 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:37.716581 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:37.741585 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:37.772238 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:37.846004 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:37.897315 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:38.031527 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:38.086048 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:38.161407 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:38.187819 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:38.215968 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:38.244802 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:38.322742 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:38.373028 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:38.506024 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:38.564605 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:38.647185 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:38.665781 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:38.693423 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:38.724771 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:38.807765 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:38.855211 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:38.991197 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:39.053653 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:39.136348 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:39.154733 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:39.179394 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:39.217182 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:39.299903 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:39.342551 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:39.485422 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:39.544569 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:39.634886 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:39.650249 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:39.678963 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:39.709224 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:39.803426 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:39.847710 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:39.991714 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:40.049219 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:40.143929 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:40.163344 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:40.189117 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:40.220048 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:40.315375 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:40.360013 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:40.508561 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:40.565008 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:40.664978 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:40.678401 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:40.709123 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:40.738790 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:40.835767 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:40.884048 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:41.028659 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:41.088445 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:41.189090 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:41.203955 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:41.236752 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:41.262305 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:41.366823 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:41.412204 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:41.560305 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:41.617543 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:41.725250 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:41.736694 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:41.771679 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:41.795449 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:41.905495 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:41.949935 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:42.095757 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:42.156651 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:42.267360 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:42.279678 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:42.315757 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:42.336585 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:42.451427 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:42.496766 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:42.644558 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:42.706797 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:42.822249 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:42.829324 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:42.871513 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:42.893302 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:43.007553 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:43.057781 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:43.203819 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:43.270680 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:43.383438 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:43.389140 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:43.432612 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:43.460250 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:43.573446 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:43.621670 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:43.771381 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:43.844357 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:43.959734 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:43.960227 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:44.005153 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:44.034857 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:44.153656 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:44.198088 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:44.347092 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:44.425731 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:44.542572 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:44.545183 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:44.590938 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:44.618654 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:44.742635 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:44.785828 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:44.936048 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:45.011852 18746 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54642: Illegal state: replica 8ba557a87917421da986e5cdc07833ad is not leader of this config: current role FOLLOWER
W20251213 08:57:45.132411 18672 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:45.134657 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:45.181704 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:45.206498 18671 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:50552: Illegal state: replica 04d79b9a9172401eb02e16f67cfa050b is not leader of this config: current role FOLLOWER
W20251213 08:57:45.330087 18826 batcher.cc:441] Timed out: Failed to write batch of 50 ops to tablet 9149b1ceecc14d1298b47ea50058dc7c after 192 attempt(s): Failed to write to server: (no server available): Write(tablet: 9149b1ceecc14d1298b47ea50058dc7c, num_ops: 50, num_attempts: 192) passed its deadline: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:57:45.408556 18086 tablet_server.cc:178] TabletServer@127.17.169.129:0 shutting down...
I20251213 08:57:45.450865 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:57:45.451637 18086 tablet_replica.cc:333] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6: stopping tablet replica
I20251213 08:57:45.452337 18086 raft_consensus.cc:2243] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:57:45.453771 18086 raft_consensus.cc:2272] T 9149b1ceecc14d1298b47ea50058dc7c P a074fef9a84040058bd02b7afcc60da6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:57:45.492658 18086 tablet_server.cc:195] TabletServer@127.17.169.129:0 shutdown complete.
I20251213 08:57:45.510762 18086 tablet_server.cc:178] TabletServer@127.17.169.130:0 shutting down...
I20251213 08:57:45.539614 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:57:45.540669 18086 tablet_replica.cc:333] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b: stopping tablet replica
I20251213 08:57:45.541319 18086 raft_consensus.cc:2243] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:57:45.542197 18086 raft_consensus.cc:2272] T 9149b1ceecc14d1298b47ea50058dc7c P 04d79b9a9172401eb02e16f67cfa050b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:57:45.564870 18086 tablet_server.cc:195] TabletServer@127.17.169.130:0 shutdown complete.
I20251213 08:57:45.583580 18086 tablet_server.cc:178] TabletServer@127.17.169.131:0 shutting down...
I20251213 08:57:45.609263 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:57:45.610095 18086 tablet_replica.cc:333] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad: stopping tablet replica
I20251213 08:57:45.610808 18086 raft_consensus.cc:2243] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:57:45.611568 18086 raft_consensus.cc:2272] T 9149b1ceecc14d1298b47ea50058dc7c P 8ba557a87917421da986e5cdc07833ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:57:45.634040 18086 tablet_server.cc:195] TabletServer@127.17.169.131:0 shutdown complete.
I20251213 08:57:45.651726 18086 master.cc:561] Master@127.17.169.190:38027 shutting down...
I20251213 08:57:45.675280 18086 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:57:45.676074 18086 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:57:45.676438 18086 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8df39f3a2a8346d79c6fb8ca9e662354: stopping tablet replica
I20251213 08:57:45.697463 18086 master.cc:583] Master@127.17.169.190:38027 shutdown complete.
[ OK ] StopTablets/StopTabletITest.TestStoppedTabletsScansGetRedirected/0 (29789 ms)
[ RUN ] StopTablets/StopTabletITest.TestStoppedTabletsScansGetRedirected/1
I20251213 08:57:45.736560 18086 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.17.169.190:35613
I20251213 08:57:45.737730 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:45.743808 18929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:45.744639 18930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:45.746090 18932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:45.746889 18086 server_base.cc:1047] running on GCE node
I20251213 08:57:45.747706 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:45.747898 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:45.748050 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616265748036 us; error 0 us; skew 500 ppm
I20251213 08:57:45.748548 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:45.750979 18086 webserver.cc:492] Webserver started at http://127.17.169.190:38135/ using document root <none> and password file <none>
I20251213 08:57:45.751446 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:45.751613 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:45.751870 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:45.753204 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/master-0-root/instance:
uuid: "26ae1766fce0466293f89069fcc3ae04"
format_stamp: "Formatted at 2025-12-13 08:57:45 on dist-test-slave-s4vp"
I20251213 08:57:45.757510 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20251213 08:57:45.760617 18937 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:45.761376 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251213 08:57:45.761626 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/master-0-root
uuid: "26ae1766fce0466293f89069fcc3ae04"
format_stamp: "Formatted at 2025-12-13 08:57:45 on dist-test-slave-s4vp"
I20251213 08:57:45.761900 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:45.779434 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:45.780593 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:45.818404 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.190:35613
I20251213 08:57:45.818487 18988 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.190:35613 every 8 connection(s)
I20251213 08:57:45.822427 18989 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:45.832568 18989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04: Bootstrap starting.
I20251213 08:57:45.837067 18989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:45.841020 18989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04: No bootstrap required, opened a new log
I20251213 08:57:45.843096 18989 raft_consensus.cc:359] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae1766fce0466293f89069fcc3ae04" member_type: VOTER }
I20251213 08:57:45.843504 18989 raft_consensus.cc:385] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:45.843739 18989 raft_consensus.cc:740] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26ae1766fce0466293f89069fcc3ae04, State: Initialized, Role: FOLLOWER
I20251213 08:57:45.844261 18989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae1766fce0466293f89069fcc3ae04" member_type: VOTER }
I20251213 08:57:45.844686 18989 raft_consensus.cc:399] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:57:45.844910 18989 raft_consensus.cc:493] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:57:45.845211 18989 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:45.849371 18989 raft_consensus.cc:515] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae1766fce0466293f89069fcc3ae04" member_type: VOTER }
I20251213 08:57:45.849850 18989 leader_election.cc:304] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26ae1766fce0466293f89069fcc3ae04; no voters:
I20251213 08:57:45.850975 18989 leader_election.cc:290] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:57:45.851373 18992 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:57:45.852634 18992 raft_consensus.cc:697] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 1 LEADER]: Becoming Leader. State: Replica: 26ae1766fce0466293f89069fcc3ae04, State: Running, Role: LEADER
I20251213 08:57:45.853294 18992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae1766fce0466293f89069fcc3ae04" member_type: VOTER }
I20251213 08:57:45.853893 18989 sys_catalog.cc:565] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [sys.catalog]: configured and running, proceeding with master startup.
I20251213 08:57:45.857743 18993 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26ae1766fce0466293f89069fcc3ae04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae1766fce0466293f89069fcc3ae04" member_type: VOTER } }
I20251213 08:57:45.858494 18993 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [sys.catalog]: This master's current role is: LEADER
I20251213 08:57:45.858333 18994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26ae1766fce0466293f89069fcc3ae04. Latest consensus state: current_term: 1 leader_uuid: "26ae1766fce0466293f89069fcc3ae04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26ae1766fce0466293f89069fcc3ae04" member_type: VOTER } }
I20251213 08:57:45.859215 18994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [sys.catalog]: This master's current role is: LEADER
I20251213 08:57:45.860595 18999 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251213 08:57:45.865671 18999 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251213 08:57:45.869864 18086 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251213 08:57:45.874529 18999 catalog_manager.cc:1357] Generated new cluster ID: 1fae880516fa4667ae8fec1ddb2d51ad
I20251213 08:57:45.874823 18999 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251213 08:57:45.886734 18999 catalog_manager.cc:1380] Generated new certificate authority record
I20251213 08:57:45.887996 18999 catalog_manager.cc:1514] Loading token signing keys...
I20251213 08:57:45.901041 18999 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04: Generated new TSK 0
I20251213 08:57:45.901593 18999 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251213 08:57:45.936273 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:45.942382 19010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:45.943711 19011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:45.945140 19013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:45.945624 18086 server_base.cc:1047] running on GCE node
I20251213 08:57:45.946537 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:45.946735 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:45.946880 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616265946863 us; error 0 us; skew 500 ppm
I20251213 08:57:45.947379 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:45.949575 18086 webserver.cc:492] Webserver started at http://127.17.169.129:35059/ using document root <none> and password file <none>
I20251213 08:57:45.950042 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:45.950251 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:45.950505 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:45.951630 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-0-root/instance:
uuid: "7f71ef3ce3b449b293869053df10748a"
format_stamp: "Formatted at 2025-12-13 08:57:45 on dist-test-slave-s4vp"
I20251213 08:57:45.956007 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20251213 08:57:45.959096 19018 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:45.959872 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20251213 08:57:45.960145 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-0-root
uuid: "7f71ef3ce3b449b293869053df10748a"
format_stamp: "Formatted at 2025-12-13 08:57:45 on dist-test-slave-s4vp"
I20251213 08:57:45.960399 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:45.976430 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:45.977593 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:45.979007 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:57:45.981146 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:57:45.981333 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:45.981554 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:57:45.981719 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:46.021435 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.129:41513
I20251213 08:57:46.021544 19080 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.129:41513 every 8 connection(s)
I20251213 08:57:46.025895 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:46.033457 19085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:46.035744 19086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:46.038570 18086 server_base.cc:1047] running on GCE node
W20251213 08:57:46.038829 19088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:46.039989 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:46.040205 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:46.040382 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616266040364 us; error 0 us; skew 500 ppm
I20251213 08:57:46.041090 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:46.041752 19081 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35613
I20251213 08:57:46.042116 19081 heartbeater.cc:461] Registering TS with master...
I20251213 08:57:46.042910 19081 heartbeater.cc:507] Master 127.17.169.190:35613 requested a full tablet report, sending...
I20251213 08:57:46.043773 18086 webserver.cc:492] Webserver started at http://127.17.169.130:44659/ using document root <none> and password file <none>
I20251213 08:57:46.044270 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:46.044510 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:46.044848 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:46.045145 18954 ts_manager.cc:194] Registered new tserver with Master: 7f71ef3ce3b449b293869053df10748a (127.17.169.129:41513)
I20251213 08:57:46.046344 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-1-root/instance:
uuid: "624f21566b334ac484539840b9ee04b7"
format_stamp: "Formatted at 2025-12-13 08:57:46 on dist-test-slave-s4vp"
I20251213 08:57:46.047652 18954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52648
I20251213 08:57:46.051429 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20251213 08:57:46.054850 19093 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:46.055716 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251213 08:57:46.055961 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-1-root
uuid: "624f21566b334ac484539840b9ee04b7"
format_stamp: "Formatted at 2025-12-13 08:57:46 on dist-test-slave-s4vp"
I20251213 08:57:46.056203 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:46.085901 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:46.087194 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:46.088667 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:57:46.090849 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:57:46.091023 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:46.091276 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:57:46.091414 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:46.131042 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.130:36799
I20251213 08:57:46.131136 19155 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.130:36799 every 8 connection(s)
I20251213 08:57:46.135643 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:57:46.142459 19159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:46.144275 19160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:57:46.146888 19163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:57:46.148064 19156 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35613
I20251213 08:57:46.148227 18086 server_base.cc:1047] running on GCE node
I20251213 08:57:46.148490 19156 heartbeater.cc:461] Registering TS with master...
I20251213 08:57:46.149089 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:57:46.149299 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:57:46.149217 19156 heartbeater.cc:507] Master 127.17.169.190:35613 requested a full tablet report, sending...
I20251213 08:57:46.149537 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616266149519 us; error 0 us; skew 500 ppm
I20251213 08:57:46.150187 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:57:46.151232 18954 ts_manager.cc:194] Registered new tserver with Master: 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799)
I20251213 08:57:46.152531 18954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52654
I20251213 08:57:46.152908 18086 webserver.cc:492] Webserver started at http://127.17.169.131:34035/ using document root <none> and password file <none>
I20251213 08:57:46.153493 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:57:46.153733 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:57:46.154054 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:57:46.155393 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-2-root/instance:
uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7"
format_stamp: "Formatted at 2025-12-13 08:57:46 on dist-test-slave-s4vp"
I20251213 08:57:46.159750 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20251213 08:57:46.162966 19167 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:46.163793 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251213 08:57:46.164048 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-2-root
uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7"
format_stamp: "Formatted at 2025-12-13 08:57:46 on dist-test-slave-s4vp"
I20251213 08:57:46.164304 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsScansGetRedirected_1.1765616224759991-18086-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:57:46.176641 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:57:46.177832 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:57:46.179344 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:57:46.181528 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:57:46.181735 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:46.181979 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:57:46.182132 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:57:46.222020 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.131:38495
I20251213 08:57:46.222111 19229 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.131:38495 every 8 connection(s)
I20251213 08:57:46.235083 19230 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35613
I20251213 08:57:46.235409 19230 heartbeater.cc:461] Registering TS with master...
I20251213 08:57:46.236045 19230 heartbeater.cc:507] Master 127.17.169.190:35613 requested a full tablet report, sending...
I20251213 08:57:46.237900 18954 ts_manager.cc:194] Registered new tserver with Master: a14ab0b5da464dc0ac054ed1b06ff4d7 (127.17.169.131:38495)
I20251213 08:57:46.238700 18086 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013506233s
I20251213 08:57:46.239439 18954 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:52666
I20251213 08:57:46.271234 18086 test_util.cc:276] Using random seed: -1839441430
I20251213 08:57:46.292713 18954 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52680:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251213 08:57:46.294615 18954 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251213 08:57:46.332652 19121 tablet_service.cc:1505] Processing CreateTablet for tablet 47a40ac0a5cf4a1db016c11fbd612775 (DEFAULT_TABLE table=test-workload [id=c6bcf0653d214cf4bdac01a3a99c9872]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:57:46.334013 19121 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47a40ac0a5cf4a1db016c11fbd612775. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:46.338933 19046 tablet_service.cc:1505] Processing CreateTablet for tablet 47a40ac0a5cf4a1db016c11fbd612775 (DEFAULT_TABLE table=test-workload [id=c6bcf0653d214cf4bdac01a3a99c9872]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:57:46.340308 19046 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47a40ac0a5cf4a1db016c11fbd612775. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:46.339990 19195 tablet_service.cc:1505] Processing CreateTablet for tablet 47a40ac0a5cf4a1db016c11fbd612775 (DEFAULT_TABLE table=test-workload [id=c6bcf0653d214cf4bdac01a3a99c9872]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:57:46.341401 19195 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47a40ac0a5cf4a1db016c11fbd612775. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:57:46.348905 19251 tablet_bootstrap.cc:492] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7: Bootstrap starting.
I20251213 08:57:46.355338 19251 tablet_bootstrap.cc:654] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:46.358060 19252 tablet_bootstrap.cc:492] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a: Bootstrap starting.
I20251213 08:57:46.359637 19253 tablet_bootstrap.cc:492] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7: Bootstrap starting.
I20251213 08:57:46.363371 19251 tablet_bootstrap.cc:492] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7: No bootstrap required, opened a new log
I20251213 08:57:46.364037 19251 ts_tablet_manager.cc:1403] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20251213 08:57:46.366179 19252 tablet_bootstrap.cc:654] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:46.366209 19253 tablet_bootstrap.cc:654] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7: Neither blocks nor log segments found. Creating new log.
I20251213 08:57:46.366891 19251 raft_consensus.cc:359] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:57:46.367697 19251 raft_consensus.cc:385] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:46.368007 19251 raft_consensus.cc:740] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 624f21566b334ac484539840b9ee04b7, State: Initialized, Role: FOLLOWER
I20251213 08:57:46.369056 19251 consensus_queue.cc:260] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:57:46.373494 19252 tablet_bootstrap.cc:492] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a: No bootstrap required, opened a new log
I20251213 08:57:46.374198 19252 ts_tablet_manager.cc:1403] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s
I20251213 08:57:46.377585 19252 raft_consensus.cc:359] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:57:46.378424 19252 raft_consensus.cc:385] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:46.378738 19252 raft_consensus.cc:740] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f71ef3ce3b449b293869053df10748a, State: Initialized, Role: FOLLOWER
I20251213 08:57:46.378971 19156 heartbeater.cc:499] Master 127.17.169.190:35613 was elected leader, sending a full tablet report...
I20251213 08:57:46.379560 19252 consensus_queue.cc:260] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:57:46.380638 19251 ts_tablet_manager.cc:1434] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7: Time spent starting tablet: real 0.016s user 0.005s sys 0.009s
I20251213 08:57:46.382593 19081 heartbeater.cc:499] Master 127.17.169.190:35613 was elected leader, sending a full tablet report...
I20251213 08:57:46.382791 19252 ts_tablet_manager.cc:1434] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20251213 08:57:46.383037 19253 tablet_bootstrap.cc:492] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7: No bootstrap required, opened a new log
I20251213 08:57:46.383482 19253 ts_tablet_manager.cc:1403] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7: Time spent bootstrapping tablet: real 0.025s user 0.015s sys 0.007s
I20251213 08:57:46.386724 19253 raft_consensus.cc:359] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:57:46.387473 19253 raft_consensus.cc:385] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:57:46.387791 19253 raft_consensus.cc:740] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a14ab0b5da464dc0ac054ed1b06ff4d7, State: Initialized, Role: FOLLOWER
I20251213 08:57:46.388563 19253 consensus_queue.cc:260] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:57:46.393524 19230 heartbeater.cc:499] Master 127.17.169.190:35613 was elected leader, sending a full tablet report...
I20251213 08:57:46.394009 19253 ts_tablet_manager.cc:1434] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7: Time spent starting tablet: real 0.010s user 0.003s sys 0.007s
I20251213 08:57:46.757395 19258 raft_consensus.cc:493] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251213 08:57:46.758203 19258 raft_consensus.cc:515] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:57:46.760538 19258 leader_election.cc:290] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799), a14ab0b5da464dc0ac054ed1b06ff4d7 (127.17.169.131:38495)
I20251213 08:57:46.773404 19131 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47a40ac0a5cf4a1db016c11fbd612775" candidate_uuid: "7f71ef3ce3b449b293869053df10748a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "624f21566b334ac484539840b9ee04b7" is_pre_election: true
I20251213 08:57:46.774358 19131 raft_consensus.cc:2468] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f71ef3ce3b449b293869053df10748a in term 0.
I20251213 08:57:46.774880 19205 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47a40ac0a5cf4a1db016c11fbd612775" candidate_uuid: "7f71ef3ce3b449b293869053df10748a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" is_pre_election: true
I20251213 08:57:46.775682 19205 raft_consensus.cc:2468] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f71ef3ce3b449b293869053df10748a in term 0.
I20251213 08:57:46.775908 19019 leader_election.cc:304] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 624f21566b334ac484539840b9ee04b7, 7f71ef3ce3b449b293869053df10748a; no voters:
I20251213 08:57:46.776787 19258 raft_consensus.cc:2804] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251213 08:57:46.777184 19258 raft_consensus.cc:493] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251213 08:57:46.777477 19258 raft_consensus.cc:3060] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:46.784323 19258 raft_consensus.cc:515] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:57:46.788419 19258 leader_election.cc:290] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [CANDIDATE]: Term 1 election: Requested vote from peers 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799), a14ab0b5da464dc0ac054ed1b06ff4d7 (127.17.169.131:38495)
I20251213 08:57:46.789253 19131 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47a40ac0a5cf4a1db016c11fbd612775" candidate_uuid: "7f71ef3ce3b449b293869053df10748a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "624f21566b334ac484539840b9ee04b7"
I20251213 08:57:46.790287 19205 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "47a40ac0a5cf4a1db016c11fbd612775" candidate_uuid: "7f71ef3ce3b449b293869053df10748a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7"
I20251213 08:57:46.790143 19131 raft_consensus.cc:3060] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:46.790961 19205 raft_consensus.cc:3060] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:57:46.799578 19205 raft_consensus.cc:2468] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f71ef3ce3b449b293869053df10748a in term 1.
I20251213 08:57:46.800629 19131 raft_consensus.cc:2468] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f71ef3ce3b449b293869053df10748a in term 1.
I20251213 08:57:46.801049 19020 leader_election.cc:304] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f71ef3ce3b449b293869053df10748a, a14ab0b5da464dc0ac054ed1b06ff4d7; no voters:
I20251213 08:57:46.801964 19258 raft_consensus.cc:2804] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:57:46.802937 19258 raft_consensus.cc:697] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 1 LEADER]: Becoming Leader. State: Replica: 7f71ef3ce3b449b293869053df10748a, State: Running, Role: LEADER
I20251213 08:57:46.803701 19258 consensus_queue.cc:237] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:57:46.813831 18953 catalog_manager.cc:5654] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f71ef3ce3b449b293869053df10748a (127.17.169.129). New cstate: current_term: 1 leader_uuid: "7f71ef3ce3b449b293869053df10748a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } health_report { overall_health: HEALTHY } } }
I20251213 08:57:46.906406 18086 test_util.cc:276] Using random seed: -1838806269
I20251213 08:57:46.933665 18086 test_workload.cc:405] TestWorkload: Skipping table creation because table test-workload already exists
I20251213 08:57:46.990823 18086 stop_tablet-itest.cc:222] 0 / 10000 rows inserted
I20251213 08:57:47.037883 18943 maintenance_manager.cc:419] P 26ae1766fce0466293f89069fcc3ae04: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.003577
I20251213 08:57:47.097612 19205 raft_consensus.cc:1275] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 1 FOLLOWER]: Refusing update from remote peer 7f71ef3ce3b449b293869053df10748a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:57:47.098284 19131 raft_consensus.cc:1275] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Refusing update from remote peer 7f71ef3ce3b449b293869053df10748a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:57:47.099925 19258 consensus_queue.cc:1048] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [LEADER]: Connected to new peer: Peer: permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251213 08:57:47.101322 19262 consensus_queue.cc:1048] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [LEADER]: Connected to new peer: Peer: permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251213 08:57:47.100660 18942 maintenance_manager.cc:643] P 26ae1766fce0466293f89069fcc3ae04: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.062s user 0.049s sys 0.008s Metrics: {"bytes_written":6635,"cfile_init":1,"dirs.queue_time_us":1245,"dirs.run_cpu_time_us":336,"dirs.run_wall_time_us":858,"drs_written":1,"lbm_read_time_us":231,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2969,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5,"spinlock_wait_cycles":45312,"thread_start_us":926,"threads_started":1}
I20251213 08:57:47.104180 18943 maintenance_manager.cc:419] P 26ae1766fce0466293f89069fcc3ae04: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 490 bytes on disk
I20251213 08:57:47.106285 18942 maintenance_manager.cc:643] P 26ae1766fce0466293f89069fcc3ae04: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":398,"lbm_reads_lt_1ms":4}
W20251213 08:57:47.122244 19185 tablet_service.cc:3012] Rejecting scan request for tablet 47a40ac0a5cf4a1db016c11fbd612775: Uninitialized: clean time has not yet been initialized
W20251213 08:57:47.143878 19111 tablet_service.cc:3012] Rejecting scan request for tablet 47a40ac0a5cf4a1db016c11fbd612775: Uninitialized: clean time has not yet been initialized
I20251213 08:57:47.461076 19157 maintenance_manager.cc:419] P 624f21566b334ac484539840b9ee04b7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.030173
W20251213 08:57:47.471786 19276 meta_cache.cc:1261] Time spent looking up entry by key: real 0.130s user 0.003s sys 0.003s
I20251213 08:57:47.481040 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.030173
I20251213 08:57:47.538578 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.030173
I20251213 08:57:47.676472 19098 maintenance_manager.cc:643] P 624f21566b334ac484539840b9ee04b7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.193s user 0.092s sys 0.032s Metrics: {"bytes_written":9878,"cfile_init":1,"dirs.queue_time_us":19704,"dirs.run_cpu_time_us":389,"dirs.run_wall_time_us":929,"drs_written":1,"lbm_read_time_us":497,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2664,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":200,"thread_start_us":10563,"threads_started":1}
I20251213 08:57:47.679982 19157 maintenance_manager.cc:419] P 624f21566b334ac484539840b9ee04b7: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 1827 bytes on disk
I20251213 08:57:47.682112 19098 maintenance_manager.cc:643] P 624f21566b334ac484539840b9ee04b7: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":400,"lbm_reads_lt_1ms":4}
I20251213 08:57:47.826906 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.343s user 0.108s sys 0.029s Metrics: {"bytes_written":9878,"cfile_init":1,"dirs.queue_time_us":26862,"dirs.run_cpu_time_us":360,"dirs.run_wall_time_us":5029,"drs_written":1,"lbm_read_time_us":776,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2701,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":200,"spinlock_wait_cycles":339712,"thread_start_us":15008,"threads_started":1}
I20251213 08:57:47.830106 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 1827 bytes on disk
I20251213 08:57:47.832551 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":371,"lbm_reads_lt_1ms":4}
I20251213 08:57:48.025380 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.486s user 0.104s sys 0.047s Metrics: {"bytes_written":11245,"cfile_init":1,"dirs.queue_time_us":35696,"dirs.run_cpu_time_us":337,"dirs.run_wall_time_us":976,"drs_written":1,"lbm_read_time_us":343,"lbm_reads_lt_1ms":4,"lbm_write_time_us":10211,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":22,"peak_mem_usage":0,"rows_written":250,"spinlock_wait_cycles":328064,"thread_start_us":41084,"threads_started":1}
I20251213 08:57:48.027484 18086 stop_tablet-itest.cc:222] 850 / 10000 rows inserted
I20251213 08:57:48.028774 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 2272 bytes on disk
I20251213 08:57:48.030514 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":302,"lbm_reads_lt_1ms":4}
W20251213 08:57:48.362835 19077 debug-util.cc:398] Leaking SignalData structure 0x7b080016d520 after lost signal to thread 18089
W20251213 08:57:48.364006 19077 debug-util.cc:398] Leaking SignalData structure 0x7b080016dc00 after lost signal to thread 18988
W20251213 08:57:48.365187 19077 debug-util.cc:398] Leaking SignalData structure 0x7b0800067f80 after lost signal to thread 19080
W20251213 08:57:48.366583 19077 debug-util.cc:398] Leaking SignalData structure 0x7b080017c460 after lost signal to thread 19155
W20251213 08:57:48.367655 19077 debug-util.cc:398] Leaking SignalData structure 0x7b080018d220 after lost signal to thread 19229
W20251213 08:57:48.423092 19274 meta_cache.cc:1261] Time spent looking up entry by key: real 0.064s user 0.000s sys 0.000s
I20251213 08:57:48.688447 19157 maintenance_manager.cc:419] P 624f21566b334ac484539840b9ee04b7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.242942
I20251213 08:57:48.860517 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.242942
I20251213 08:57:49.034840 18086 stop_tablet-itest.cc:222] 2400 / 10000 rows inserted
I20251213 08:57:49.038916 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.242942
W20251213 08:57:49.750749 19277 scanner-internal.cc:458] Time spent opening tablet: real 0.651s user 0.001s sys 0.000s
I20251213 08:57:49.838013 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.977s user 0.500s sys 0.010s Metrics: {"bytes_written":57872,"cfile_init":1,"dirs.queue_time_us":17501,"dirs.run_cpu_time_us":336,"dirs.run_wall_time_us":10736,"drs_written":1,"lbm_read_time_us":281,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3289,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":1900,"thread_start_us":17743,"threads_started":1}
I20251213 08:57:49.846341 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.006459
I20251213 08:57:50.040685 18086 stop_tablet-itest.cc:222] 3300 / 10000 rows inserted
I20251213 08:57:50.318548 19098 maintenance_manager.cc:643] P 624f21566b334ac484539840b9ee04b7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 1.629s user 0.447s sys 0.063s Metrics: {"bytes_written":47376,"cfile_init":1,"dirs.queue_time_us":128666,"dirs.run_cpu_time_us":385,"dirs.run_wall_time_us":8620,"drs_written":1,"lbm_read_time_us":292,"lbm_reads_lt_1ms":4,"lbm_write_time_us":20437,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":22,"peak_mem_usage":0,"rows_written":1650,"thread_start_us":129440,"threads_started":1}
I20251213 08:57:50.322415 19157 maintenance_manager.cc:419] P 624f21566b334ac484539840b9ee04b7: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.007105
I20251213 08:57:50.505620 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.655s user 0.433s sys 0.015s Metrics: {"bytes_written":62953,"cfile_cache_hit":23,"cfile_cache_hit_bytes":60339,"cfile_cache_miss":13,"cfile_cache_miss_bytes":85603,"cfile_init":4,"delta_iterators_relevant":4,"dirs.queue_time_us":6845,"dirs.run_cpu_time_us":327,"dirs.run_wall_time_us":18216,"drs_written":1,"lbm_read_time_us":2368,"lbm_reads_lt_1ms":29,"lbm_write_time_us":2947,"lbm_writes_lt_1ms":24,"num_input_rowsets":2,"peak_mem_usage":36062,"rows_written":2100,"thread_start_us":7047,"threads_started":1}
I20251213 08:57:50.508888 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 18360 bytes on disk
I20251213 08:57:50.510656 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":266,"lbm_reads_lt_1ms":4}
W20251213 08:57:50.659086 19277 scanner-internal.cc:458] Time spent opening tablet: real 0.724s user 0.001s sys 0.000s
I20251213 08:57:50.723745 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 1.684s user 0.595s sys 0.026s Metrics: {"bytes_written":65405,"cfile_init":1,"dirs.queue_time_us":34473,"dirs.run_cpu_time_us":351,"dirs.run_wall_time_us":4734,"drs_written":1,"lbm_read_time_us":282,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3073,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":2200,"thread_start_us":34726,"threads_started":1}
I20251213 08:57:50.749228 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 19259 bytes on disk
I20251213 08:57:50.750990 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":293,"lbm_reads_lt_1ms":4}
I20251213 08:57:50.770643 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.006522
I20251213 08:57:51.015069 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.242942
I20251213 08:57:51.044495 18086 stop_tablet-itest.cc:222] 4350 / 10000 rows inserted
I20251213 08:57:51.283222 19098 maintenance_manager.cc:643] P 624f21566b334ac484539840b9ee04b7: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.960s user 0.437s sys 0.045s Metrics: {"bytes_written":56548,"cfile_cache_hit":10,"cfile_cache_hit_bytes":33976,"cfile_cache_miss":26,"cfile_cache_miss_bytes":94718,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":9950,"dirs.run_cpu_time_us":1676,"dirs.run_wall_time_us":37227,"drs_written":1,"lbm_read_time_us":3978,"lbm_reads_lt_1ms":50,"lbm_write_time_us":3074,"lbm_writes_lt_1ms":24,"num_input_rowsets":2,"peak_mem_usage":30852,"rows_written":1850,"spinlock_wait_cycles":57984,"thread_start_us":80685,"threads_started":3}
I20251213 08:57:51.286672 19157 maintenance_manager.cc:419] P 624f21566b334ac484539840b9ee04b7: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 16112 bytes on disk
I20251213 08:57:51.288477 19098 maintenance_manager.cc:643] P 624f21566b334ac484539840b9ee04b7: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":277,"lbm_reads_lt_1ms":4}
I20251213 08:57:51.541339 19157 maintenance_manager.cc:419] P 624f21566b334ac484539840b9ee04b7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.486106
W20251213 08:57:51.807039 19277 scanner-internal.cc:458] Time spent opening tablet: real 0.734s user 0.001s sys 0.000s
I20251213 08:57:52.050902 18086 stop_tablet-itest.cc:222] 5400 / 10000 rows inserted
I20251213 08:57:52.372213 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 1.356s user 0.636s sys 0.018s Metrics: {"bytes_written":66736,"cfile_init":1,"dirs.queue_time_us":53964,"dirs.run_cpu_time_us":307,"dirs.run_wall_time_us":3859,"drs_written":1,"lbm_read_time_us":365,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3257,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":2250,"thread_start_us":53657,"threads_started":1}
I20251213 08:57:52.380577 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.014452
I20251213 08:57:52.374436 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 1.603s user 0.521s sys 0.020s Metrics: {"bytes_written":71940,"cfile_cache_hit":10,"cfile_cache_hit_bytes":44927,"cfile_cache_miss":26,"cfile_cache_miss_bytes":125153,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":23181,"dirs.run_cpu_time_us":335,"dirs.run_wall_time_us":19120,"drs_written":1,"lbm_read_time_us":16920,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":45,"lbm_write_time_us":3322,"lbm_writes_lt_1ms":25,"num_input_rowsets":2,"peak_mem_usage":39361,"rows_written":2450,"thread_start_us":22862,"threads_started":1}
I20251213 08:57:52.382978 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.486106
I20251213 08:57:53.065245 18086 stop_tablet-itest.cc:222] 6200 / 10000 rows inserted
W20251213 08:57:53.304222 19277 scanner-internal.cc:458] Time spent opening tablet: real 0.865s user 0.002s sys 0.000s
W20251213 08:57:53.379328 19276 meta_cache.cc:1261] Time spent looking up entry by key: real 0.054s user 0.002s sys 0.000s
I20251213 08:57:53.601780 19098 maintenance_manager.cc:643] P 624f21566b334ac484539840b9ee04b7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 2.060s user 0.817s sys 0.017s Metrics: {"bytes_written":87458,"cfile_init":1,"dirs.queue_time_us":10365,"dirs.run_cpu_time_us":330,"dirs.run_wall_time_us":139637,"drs_written":1,"lbm_read_time_us":288,"lbm_reads_lt_1ms":4,"lbm_write_time_us":26426,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":3050,"thread_start_us":10585,"threads_started":1}
I20251213 08:57:53.656258 19157 maintenance_manager.cc:419] P 624f21566b334ac484539840b9ee04b7: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.014463
I20251213 08:57:53.842815 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 1.456s user 0.877s sys 0.012s Metrics: {"bytes_written":94824,"cfile_init":1,"dirs.queue_time_us":12288,"dirs.run_cpu_time_us":301,"dirs.run_wall_time_us":9974,"drs_written":1,"lbm_read_time_us":262,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3386,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3350,"thread_start_us":12413,"threads_started":1}
I20251213 08:57:53.846318 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 51824 bytes on disk
I20251213 08:57:53.848655 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":532,"lbm_reads_lt_1ms":8}
I20251213 08:57:53.851307 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.014814
I20251213 08:57:53.988677 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 1.605s user 0.853s sys 0.027s Metrics: {"bytes_written":125057,"cfile_cache_hit":23,"cfile_cache_hit_bytes":167118,"cfile_cache_miss":13,"cfile_cache_miss_bytes":134008,"cfile_init":4,"delta_iterators_relevant":4,"dirs.queue_time_us":5727,"dirs.run_cpu_time_us":304,"dirs.run_wall_time_us":8118,"drs_written":1,"lbm_read_time_us":2598,"lbm_reads_lt_1ms":29,"lbm_write_time_us":3843,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":66436,"rows_written":4350,"thread_start_us":25107,"threads_started":1}
I20251213 08:57:54.017870 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.486106
I20251213 08:57:54.078908 18086 stop_tablet-itest.cc:222] 6900 / 10000 rows inserted
I20251213 08:57:55.082290 18086 stop_tablet-itest.cc:222] 7850 / 10000 rows inserted
W20251213 08:57:55.589533 19277 scanner-internal.cc:458] Time spent opening tablet: real 0.823s user 0.001s sys 0.000s
I20251213 08:57:55.660566 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 1.641s user 0.668s sys 0.024s Metrics: {"bytes_written":73159,"cfile_init":1,"dirs.queue_time_us":93,"dirs.run_cpu_time_us":224,"dirs.run_wall_time_us":1292,"drs_written":1,"lbm_read_time_us":281,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3226,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2500}
I20251213 08:57:55.665964 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.014759
I20251213 08:57:56.094473 18086 stop_tablet-itest.cc:222] 8800 / 10000 rows inserted
I20251213 08:57:56.450668 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 2.599s user 1.163s sys 0.042s Metrics: {"bytes_written":167584,"cfile_cache_hit":10,"cfile_cache_hit_bytes":106113,"cfile_cache_miss":30,"cfile_cache_miss_bytes":296543,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":10424,"dirs.run_cpu_time_us":363,"dirs.run_wall_time_us":8119,"drs_written":1,"lbm_read_time_us":12147,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":49,"lbm_write_time_us":16505,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":86661,"rows_written":5800,"spinlock_wait_cycles":2816,"thread_start_us":10046,"threads_started":1}
I20251213 08:57:56.454799 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.486106
I20251213 08:57:57.029947 19098 maintenance_manager.cc:643] P 624f21566b334ac484539840b9ee04b7: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 3.373s user 1.014s sys 0.052s Metrics: {"bytes_written":139751,"cfile_cache_hit":10,"cfile_cache_hit_bytes":89665,"cfile_cache_miss":28,"cfile_cache_miss_bytes":250797,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":40671,"dirs.run_cpu_time_us":1661,"dirs.run_wall_time_us":8646,"drs_written":1,"lbm_read_time_us":4628,"lbm_reads_lt_1ms":52,"lbm_write_time_us":4390,"lbm_writes_lt_1ms":27,"mutex_wait_us":243,"num_input_rowsets":2,"peak_mem_usage":73381,"rows_written":4900,"thread_start_us":126921,"threads_started":3}
I20251213 08:57:57.039660 19157 maintenance_manager.cc:419] P 624f21566b334ac484539840b9ee04b7: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 44655 bytes on disk
I20251213 08:57:57.048841 19098 maintenance_manager.cc:643] P 624f21566b334ac484539840b9ee04b7: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":290,"lbm_reads_lt_1ms":4}
I20251213 08:57:57.051276 19157 maintenance_manager.cc:419] P 624f21566b334ac484539840b9ee04b7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.972434
I20251213 08:57:57.109114 18086 stop_tablet-itest.cc:222] 9700 / 10000 rows inserted
W20251213 08:57:57.109735 19274 meta_cache.cc:1261] Time spent looking up entry by key: real 0.060s user 0.003s sys 0.000s
W20251213 08:57:57.423421 19273 meta_cache.cc:1261] Time spent looking up entry by key: real 0.066s user 0.001s sys 0.000s
I20251213 08:57:57.501837 18086 stop_tablet-itest.cc:98] Stopping T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7
W20251213 08:57:57.503409 19294 op_driver.cc:548] Did not Apply op WriteOp [type=FOLLOWER, start_time=2025-12-13 08:57:57, state=WriteOpState 0x7b5800299700 [op_id=(term: 1 index: 202), ts=7231964271827456000, rows=[INSERT (int32 key=332564612, int32 int_val=1024715377, string string_val="0aae028dcc533a91"), INSERT (int32 key=1155473950, int32 int_val=-772992187, string string_val="e680c3399f220f84"), INSERT (int32 key=1124393084, int32 int_val=-1561578545, string string_val="93ddd2aa1063422d"), ...]]]: Illegal state: Apply of WriteOpState 0x7b5800299700 [op_id=(term: 1 index: 202), ts=7231964271827456000, rows=[INSERT (int32 key=332564612, int32 int_val=1024715377, string string_val="0aae028dcc533a91"), INSERT (int32 key=1155473950, int32 int_val=-772992187, string string_val="e680c3399f220f84"), INSERT (int32 key=1124393084, int32 int_val=-1561578545, string string_val="93ddd2aa1063422d"), ...]] exited early: Tablet has been stopped
W20251213 08:57:57.504467 19294 mvcc.cc:118] aborting op with timestamp 7231964271827456000 in state 1; MVCC is closed
W20251213 08:57:57.506140 19294 mvcc.cc:118] aborting op with timestamp 7231964272010502144 in state 0; MVCC is closed
W20251213 08:57:57.507512 19294 mvcc.cc:118] aborting op with timestamp 7231964272480022528 in state 0; MVCC is closed
W20251213 08:57:57.515395 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 1 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:57:57.518492 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:57:57.519912 19019 consensus_peers.cc:597] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a -> Peer 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799): Couldn't send request to peer 624f21566b334ac484539840b9ee04b7. Status: Illegal state: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped. This is attempt 1: this message will repeat every 5th retry.
W20251213 08:57:57.582849 19349 op_driver.cc:548] Did not Apply op WriteOp [type=FOLLOWER, start_time=2025-12-13 08:57:57, state=WriteOpState 0x7b5800299d00 [op_id=(term: 1 index: 201), ts=7231964271681212416, rows=[INSERT (int32 key=1631610743, int32 int_val=525807211, string string_val="2194255c0a41987a"), INSERT (int32 key=463572045, int32 int_val=-1193347370, string string_val="6618f4a2e0c9d817"), INSERT (int32 key=2028919984, int32 int_val=-2115640241, string string_val="14b7df11672bbf6e"), ...]]]: Illegal state: Apply of WriteOpState 0x7b5800299d00 [op_id=(term: 1 index: 201), ts=7231964271681212416, rows=[INSERT (int32 key=1631610743, int32 int_val=525807211, string string_val="2194255c0a41987a"), INSERT (int32 key=463572045, int32 int_val=-1193347370, string string_val="6618f4a2e0c9d817"), INSERT (int32 key=2028919984, int32 int_val=-2115640241, string string_val="14b7df11672bbf6e"), ...]] exited early: Tablet has been stopped
W20251213 08:57:57.598484 19349 mvcc.cc:118] aborting op with timestamp 7231964271681212416 in state 1; MVCC is closed
W20251213 08:57:57.600939 19312 op_driver.cc:548] Did not Apply op WriteOp [type=FOLLOWER, start_time=2025-12-13 08:57:57, state=WriteOpState 0x7b580035d300 [op_id=(term: 1 index: 198), ts=7231964271243341824, rows=[INSERT (int32 key=1434573147, int32 int_val=1473090637, string string_val="08065a94b2499efa"), INSERT (int32 key=283945847, int32 int_val=-715503121, string string_val="cbe1f1bf3d8c7162"), INSERT (int32 key=854295187, int32 int_val=-1461722138, string string_val="ce0606935b33482c"), ...]]]: Illegal state: Apply of WriteOpState 0x7b580035d300 [op_id=(term: 1 index: 198), ts=7231964271243341824, rows=[INSERT (int32 key=1434573147, int32 int_val=1473090637, string string_val="08065a94b2499efa"), INSERT (int32 key=283945847, int32 int_val=-715503121, string string_val="cbe1f1bf3d8c7162"), INSERT (int32 key=854295187, int32 int_val=-1461722138, string string_val="ce0606935b33482c"), ...]] exited early: Tablet has been stopped
W20251213 08:57:57.591091 19350 op_driver.cc:548] Did not Apply op WriteOp [type=FOLLOWER, start_time=2025-12-13 08:57:57, state=WriteOpState 0x7b580035d600 [op_id=(term: 1 index: 199), ts=7231964271385923584, rows=[INSERT (int32 key=1856366175, int32 int_val=-766029173, string string_val="c3f7768ad70f01c4"), INSERT (int32 key=550676202, int32 int_val=-871536013, string string_val="6c711d826c90b132"), INSERT (int32 key=1248849438, int32 int_val=-433026710, string string_val="1f703d158374bd4d"), ...]]]: Illegal state: Apply of WriteOpState 0x7b580035d600 [op_id=(term: 1 index: 199), ts=7231964271385923584, rows=[INSERT (int32 key=1856366175, int32 int_val=-766029173, string string_val="c3f7768ad70f01c4"), INSERT (int32 key=550676202, int32 int_val=-871536013, string string_val="6c711d826c90b132"), INSERT (int32 key=1248849438, int32 int_val=-433026710, string string_val="1f703d158374bd4d"), ...]] exited early: Tablet has been stopped
W20251213 08:57:57.606385 19312 mvcc.cc:118] aborting op with timestamp 7231964271243341824 in state 1; MVCC is closed
W20251213 08:57:57.606832 19350 mvcc.cc:118] aborting op with timestamp 7231964271385923584 in state 1; MVCC is closed
W20251213 08:57:57.839954 19277 scanner-internal.cc:458] Time spent opening tablet: real 0.713s user 0.001s sys 0.000s
W20251213 08:57:57.912379 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 9 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:57:57.936237 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:57:58.104333 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 1.649s user 0.915s sys 0.013s Metrics: {"bytes_written":95991,"cfile_init":1,"dirs.queue_time_us":95003,"dirs.run_cpu_time_us":343,"dirs.run_wall_time_us":10962,"drs_written":1,"lbm_read_time_us":288,"lbm_reads_lt_1ms":4,"lbm_write_time_us":13533,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":3400,"thread_start_us":112381,"threads_started":1}
I20251213 08:57:58.107933 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 83416 bytes on disk
I20251213 08:57:58.123955 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.007s user 0.002s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":1218,"lbm_reads_lt_1ms":8}
I20251213 08:57:58.126781 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.014763
W20251213 08:57:58.309932 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 19 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:57:58.311375 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:57:58.813030 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 28 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:57:58.814719 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:57:59.078850 19275 meta_cache.cc:1261] Time spent looking up entry by key: real 0.057s user 0.003s sys 0.000s
I20251213 08:57:59.171087 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 3.501s user 1.328s sys 0.044s Metrics: {"bytes_written":199437,"cfile_cache_hit":23,"cfile_cache_hit_bytes":289556,"cfile_cache_miss":17,"cfile_cache_miss_bytes":187038,"cfile_init":4,"delta_iterators_relevant":4,"dirs.queue_time_us":12145,"dirs.run_cpu_time_us":308,"dirs.run_wall_time_us":8622,"drs_written":1,"lbm_read_time_us":3151,"lbm_reads_lt_1ms":33,"lbm_write_time_us":19848,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":29,"num_input_rowsets":2,"peak_mem_usage":104321,"rows_written":6850,"thread_start_us":6742,"threads_started":1}
I20251213 08:57:59.174378 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 63139 bytes on disk
I20251213 08:57:59.188009 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":378,"lbm_reads_lt_1ms":4}
I20251213 08:57:59.190232 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.972434
W20251213 08:57:59.220392 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 38 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:57:59.241580 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:57:59.735018 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 47 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:57:59.739961 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:57:59.743925 19019 consensus_peers.cc:597] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a -> Peer 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799): Couldn't send request to peer 624f21566b334ac484539840b9ee04b7. Status: Illegal state: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped. This is attempt 6: this message will repeat every 5th retry.
W20251213 08:57:59.992444 19098 tablet_replica_mm_ops.cc:214] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7: failed to flush MRS: Aborted: MVCC is closed
I20251213 08:57:59.992933 19098 maintenance_manager.cc:643] P 624f21566b334ac484539840b9ee04b7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 2.941s user 0.982s sys 0.020s Metrics: {"cfile_init":1,"dirs.queue_time_us":5901,"dirs.run_cpu_time_us":304,"dirs.run_wall_time_us":1391,"lbm_read_time_us":302,"lbm_reads_lt_1ms":4,"lbm_write_time_us":29603,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":25,"thread_start_us":9177,"threads_started":1}
W20251213 08:58:00.116724 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 56 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:00.124604 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:00.183389 19274 meta_cache.cc:1261] Time spent looking up entry by key: real 0.061s user 0.002s sys 0.000s
W20251213 08:58:00.624595 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 63 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:00.636770 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:01.065049 19277 scanner-internal.cc:458] Time spent opening tablet: real 0.676s user 0.001s sys 0.000s
W20251213 08:58:01.234431 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 75 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:01.238248 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:01.407938 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 2.217s user 1.415s sys 0.012s Metrics: {"bytes_written":151694,"cfile_init":1,"dirs.queue_time_us":6912,"dirs.run_cpu_time_us":289,"dirs.run_wall_time_us":5437,"drs_written":1,"lbm_read_time_us":271,"lbm_reads_lt_1ms":4,"lbm_write_time_us":8578,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5200,"spinlock_wait_cycles":234240,"thread_start_us":7084,"threads_started":1}
I20251213 08:58:01.412966 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.014701
I20251213 08:58:01.451951 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 3.323s user 1.828s sys 0.048s Metrics: {"bytes_written":265954,"cfile_cache_hit":12,"cfile_cache_hit_bytes":199772,"cfile_cache_miss":30,"cfile_cache_miss_bytes":443212,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":19544,"dirs.run_cpu_time_us":429,"dirs.run_wall_time_us":9805,"drs_written":1,"lbm_read_time_us":33307,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":48,"lbm_write_time_us":5793,"lbm_writes_lt_1ms":32,"num_input_rowsets":2,"peak_mem_usage":141804,"rows_written":9200,"spinlock_wait_cycles":34560,"thread_start_us":24681,"threads_started":1}
I20251213 08:58:01.460155 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.972434
W20251213 08:58:01.853829 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 84 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:01.857496 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:02.929945 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 96 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:02.934443 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:02.938047 19019 consensus_peers.cc:597] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a -> Peer 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799): Couldn't send request to peer 624f21566b334ac484539840b9ee04b7. Status: Illegal state: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped. This is attempt 11: this message will repeat every 5th retry.
W20251213 08:58:03.393846 19226 debug-util.cc:398] Leaking SignalData structure 0x7b08000cbe00 after lost signal to thread 18089
W20251213 08:58:03.395020 19226 debug-util.cc:398] Leaking SignalData structure 0x7b08000c84e0 after lost signal to thread 18988
W20251213 08:58:03.396119 19226 debug-util.cc:398] Leaking SignalData structure 0x7b08001aab20 after lost signal to thread 19080
W20251213 08:58:03.397045 19226 debug-util.cc:398] Leaking SignalData structure 0x7b08001ab360 after lost signal to thread 19155
W20251213 08:58:03.397946 19226 debug-util.cc:398] Leaking SignalData structure 0x7b08001abba0 after lost signal to thread 19229
W20251213 08:58:03.534494 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 108 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:03.556005 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:04.141969 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 117 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:04.178507 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:04.621011 19277 scanner-internal.cc:458] Time spent opening tablet: real 0.586s user 0.002s sys 0.000s
W20251213 08:58:04.692586 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 130 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:04.719759 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:05.086380 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 139 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:05.114394 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:05.124059 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 3.663s user 1.364s sys 0.031s Metrics: {"bytes_written":145030,"cfile_init":1,"dirs.queue_time_us":7087,"dirs.run_cpu_time_us":307,"dirs.run_wall_time_us":5324,"drs_written":1,"lbm_read_time_us":278,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4091,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5100,"thread_start_us":7327,"threads_started":1}
I20251213 08:58:05.128998 19082 maintenance_manager.cc:419] P 7f71ef3ce3b449b293869053df10748a: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.014778
W20251213 08:58:05.538700 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 148 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:05.561738 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:05.598848 19019 consensus_peers.cc:597] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a -> Peer 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799): Couldn't send request to peer 624f21566b334ac484539840b9ee04b7. Status: Illegal state: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped. This is attempt 16: this message will repeat every 5th retry.
I20251213 08:58:05.927656 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 4.512s user 2.302s sys 0.071s Metrics: {"bytes_written":349070,"cfile_cache_hit":23,"cfile_cache_hit_bytes":492202,"cfile_cache_miss":23,"cfile_cache_miss_bytes":355560,"cfile_init":4,"delta_iterators_relevant":4,"dirs.queue_time_us":31165,"dirs.run_cpu_time_us":307,"dirs.run_wall_time_us":33583,"drs_written":1,"lbm_read_time_us":5634,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":38,"lbm_write_time_us":7460,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":34,"num_input_rowsets":2,"peak_mem_usage":183610,"rows_written":12050,"thread_start_us":32989,"threads_started":1}
I20251213 08:58:05.941107 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775): 113489 bytes on disk
I20251213 08:58:05.943698 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: UndoDeltaBlockGCOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":282,"lbm_reads_lt_1ms":4}
I20251213 08:58:05.945824 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=0.972434
W20251213 08:58:06.096148 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 159 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:06.112684 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:06.658056 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 173 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:06.684258 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:07.213018 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 185 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:07.299141 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:07.527853 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 196 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:07.581456 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:07.934469 18086 tablet_server.cc:178] TabletServer@127.17.169.129:0 shutting down...
W20251213 08:58:08.006727 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 205 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:08.037259 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:08.042510 19019 consensus_peers.cc:597] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a -> Peer 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799): Couldn't send request to peer 624f21566b334ac484539840b9ee04b7. Status: Illegal state: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped. This is attempt 21: this message will repeat every 5th retry.
W20251213 08:58:08.305328 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 206 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:08.313122 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:08.403352 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: FlushMRSOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 2.448s user 1.446s sys 0.036s Metrics: {"bytes_written":180688,"cfile_init":1,"dirs.queue_time_us":111,"dirs.run_cpu_time_us":311,"dirs.run_wall_time_us":990,"drs_written":1,"lbm_read_time_us":228,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3557,"lbm_writes_lt_1ms":28,"peak_mem_usage":0,"rows_written":6300}
I20251213 08:58:08.407094 19231 maintenance_manager.cc:419] P a14ab0b5da464dc0ac054ed1b06ff4d7: Scheduling CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775): perf score=1.014716
W20251213 08:58:08.733071 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 206 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:08.741869 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:09.238740 19131 raft_consensus.cc:1579] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 205' and following 206 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:09.248256 19131 raft_consensus.cc:1595] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Rejecting Update request from peer 7f71ef3ce3b449b293869053df10748a for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:09.508142 19023 maintenance_manager.cc:643] P 7f71ef3ce3b449b293869053df10748a: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 4.378s user 2.981s sys 0.087s Metrics: {"bytes_written":412129,"cfile_cache_hit":12,"cfile_cache_hit_bytes":325213,"cfile_cache_miss":36,"cfile_cache_miss_bytes":685411,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":4560,"dirs.run_cpu_time_us":1287,"dirs.run_wall_time_us":5547,"drs_written":1,"lbm_read_time_us":15950,"lbm_reads_1-10_ms":3,"lbm_reads_lt_1ms":61,"lbm_write_time_us":20334,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":36,"num_input_rowsets":2,"peak_mem_usage":219568,"rows_written":14300,"thread_start_us":4115,"threads_started":6,"wal-append.queue_time_us":927}
I20251213 08:58:09.513921 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:58:09.515002 18086 tablet_replica.cc:333] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a: stopping tablet replica
I20251213 08:58:09.515766 18086 raft_consensus.cc:2243] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:58:09.527829 18086 raft_consensus.cc:2272] T 47a40ac0a5cf4a1db016c11fbd612775 P 7f71ef3ce3b449b293869053df10748a [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:09.599196 18086 tablet_server.cc:195] TabletServer@127.17.169.129:0 shutdown complete.
I20251213 08:58:09.617826 18086 tablet_server.cc:178] TabletServer@127.17.169.130:0 shutting down...
I20251213 08:58:09.673664 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:58:09.674552 18086 tablet_replica.cc:333] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7: stopping tablet replica
I20251213 08:58:09.675339 18086 raft_consensus.cc:2243] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:58:09.676664 18086 raft_consensus.cc:2272] T 47a40ac0a5cf4a1db016c11fbd612775 P 624f21566b334ac484539840b9ee04b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:09.733530 18086 tablet_server.cc:195] TabletServer@127.17.169.130:0 shutdown complete.
I20251213 08:58:09.750914 18086 tablet_server.cc:178] TabletServer@127.17.169.131:0 shutting down...
I20251213 08:58:10.532883 19404 raft_consensus.cc:493] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7f71ef3ce3b449b293869053df10748a)
I20251213 08:58:10.533386 19404 raft_consensus.cc:515] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "624f21566b334ac484539840b9ee04b7" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 36799 } } peers { permanent_uuid: "a14ab0b5da464dc0ac054ed1b06ff4d7" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 38495 } } peers { permanent_uuid: "7f71ef3ce3b449b293869053df10748a" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 41513 } }
I20251213 08:58:10.535715 19404 leader_election.cc:290] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799), 7f71ef3ce3b449b293869053df10748a (127.17.169.129:41513)
W20251213 08:58:10.539029 19168 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.169.130:36799: connect: Connection refused (error 111)
W20251213 08:58:10.542551 19168 leader_election.cc:336] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 624f21566b334ac484539840b9ee04b7 (127.17.169.130:36799): Network error: Client connection negotiation failed: client connection to 127.17.169.130:36799: connect: Connection refused (error 111)
W20251213 08:58:10.543289 19168 leader_election.cc:336] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7f71ef3ce3b449b293869053df10748a (127.17.169.129:41513): Network error: Client connection negotiation failed: client connection to 127.17.169.129:41513: connect: Connection refused (error 111)
I20251213 08:58:10.543682 19168 leader_election.cc:304] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a14ab0b5da464dc0ac054ed1b06ff4d7; no voters: 624f21566b334ac484539840b9ee04b7, 7f71ef3ce3b449b293869053df10748a
I20251213 08:58:10.544435 19404 raft_consensus.cc:2749] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251213 08:58:10.601220 19172 maintenance_manager.cc:643] P a14ab0b5da464dc0ac054ed1b06ff4d7: CompactRowSetsOp(47a40ac0a5cf4a1db016c11fbd612775) complete. Timing: real 2.193s user 2.138s sys 0.052s Metrics: {"bytes_written":522429,"cfile_cache_hit":10,"cfile_cache_hit_bytes":345426,"cfile_cache_miss":40,"cfile_cache_miss_bytes":955718,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":3018,"dirs.run_cpu_time_us":1205,"dirs.run_wall_time_us":5334,"drs_written":1,"lbm_read_time_us":9762,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":62,"lbm_write_time_us":6657,"lbm_writes_lt_1ms":42,"mutex_wait_us":41,"num_input_rowsets":2,"peak_mem_usage":279676,"rows_written":18350,"spinlock_wait_cycles":201088,"thread_start_us":3145,"threads_started":5,"wal-append.queue_time_us":1004}
I20251213 08:58:10.605230 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:58:10.605937 18086 tablet_replica.cc:333] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7: stopping tablet replica
I20251213 08:58:10.606467 18086 raft_consensus.cc:2243] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:58:10.615594 18086 raft_consensus.cc:2272] T 47a40ac0a5cf4a1db016c11fbd612775 P a14ab0b5da464dc0ac054ed1b06ff4d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:10.662297 18086 tablet_server.cc:195] TabletServer@127.17.169.131:0 shutdown complete.
I20251213 08:58:10.675107 18086 master.cc:561] Master@127.17.169.190:35613 shutting down...
I20251213 08:58:10.692510 18086 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:58:10.693074 18086 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:10.693367 18086 tablet_replica.cc:333] T 00000000000000000000000000000000 P 26ae1766fce0466293f89069fcc3ae04: stopping tablet replica
I20251213 08:58:10.712054 18086 master.cc:583] Master@127.17.169.190:35613 shutdown complete.
[ OK ] StopTablets/StopTabletITest.TestStoppedTabletsScansGetRedirected/1 (25011 ms)
[ RUN ] StopTablets/StopTabletITest.TestStoppedTabletsDontWrite/0
I20251213 08:58:10.748436 18086 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.17.169.190:35755
I20251213 08:58:10.749558 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:10.755210 19412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:10.755790 19413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:10.756203 19415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:10.757050 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:10.757936 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:10.758128 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:10.758456 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616290758440 us; error 0 us; skew 500 ppm
I20251213 08:58:10.758916 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:10.761183 18086 webserver.cc:492] Webserver started at http://127.17.169.190:39137/ using document root <none> and password file <none>
I20251213 08:58:10.761668 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:10.761828 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:10.762037 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:10.763130 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/master-0-root/instance:
uuid: "1ab2f27452ec4db4a09aea81623ee921"
format_stamp: "Formatted at 2025-12-13 08:58:10 on dist-test-slave-s4vp"
I20251213 08:58:10.767453 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20251213 08:58:10.771055 19420 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:10.771968 18086 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20251213 08:58:10.772260 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/master-0-root
uuid: "1ab2f27452ec4db4a09aea81623ee921"
format_stamp: "Formatted at 2025-12-13 08:58:10 on dist-test-slave-s4vp"
I20251213 08:58:10.772549 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:10.786092 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:10.787408 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:10.828770 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.190:35755
I20251213 08:58:10.828881 19471 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.190:35755 every 8 connection(s)
I20251213 08:58:10.832731 19472 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:58:10.843945 19472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921: Bootstrap starting.
I20251213 08:58:10.848461 19472 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921: Neither blocks nor log segments found. Creating new log.
I20251213 08:58:10.852531 19472 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921: No bootstrap required, opened a new log
I20251213 08:58:10.854631 19472 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab2f27452ec4db4a09aea81623ee921" member_type: VOTER }
I20251213 08:58:10.855002 19472 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:58:10.855178 19472 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ab2f27452ec4db4a09aea81623ee921, State: Initialized, Role: FOLLOWER
I20251213 08:58:10.855613 19472 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab2f27452ec4db4a09aea81623ee921" member_type: VOTER }
I20251213 08:58:10.856024 19472 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:58:10.856206 19472 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:58:10.856405 19472 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:58:10.860531 19472 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab2f27452ec4db4a09aea81623ee921" member_type: VOTER }
I20251213 08:58:10.860967 19472 leader_election.cc:304] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ab2f27452ec4db4a09aea81623ee921; no voters:
I20251213 08:58:10.862026 19472 leader_election.cc:290] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:58:10.862473 19475 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:58:10.863736 19475 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 1 LEADER]: Becoming Leader. State: Replica: 1ab2f27452ec4db4a09aea81623ee921, State: Running, Role: LEADER
I20251213 08:58:10.864394 19475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab2f27452ec4db4a09aea81623ee921" member_type: VOTER }
I20251213 08:58:10.865046 19472 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [sys.catalog]: configured and running, proceeding with master startup.
I20251213 08:58:10.868779 19476 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ab2f27452ec4db4a09aea81623ee921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab2f27452ec4db4a09aea81623ee921" member_type: VOTER } }
I20251213 08:58:10.870049 19476 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [sys.catalog]: This master's current role is: LEADER
I20251213 08:58:10.869642 19477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ab2f27452ec4db4a09aea81623ee921. Latest consensus state: current_term: 1 leader_uuid: "1ab2f27452ec4db4a09aea81623ee921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ab2f27452ec4db4a09aea81623ee921" member_type: VOTER } }
I20251213 08:58:10.871107 19477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [sys.catalog]: This master's current role is: LEADER
I20251213 08:58:10.877480 19484 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251213 08:58:10.882267 19484 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251213 08:58:10.883260 18086 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251213 08:58:10.890468 19484 catalog_manager.cc:1357] Generated new cluster ID: 3fbca2a4862845299db1791c05f8456b
I20251213 08:58:10.890797 19484 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251213 08:58:10.925400 19484 catalog_manager.cc:1380] Generated new certificate authority record
I20251213 08:58:10.926822 19484 catalog_manager.cc:1514] Loading token signing keys...
I20251213 08:58:10.940747 19484 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921: Generated new TSK 0
I20251213 08:58:10.941349 19484 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251213 08:58:10.949539 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:10.955462 19493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:10.956504 19494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:10.959079 18086 server_base.cc:1047] running on GCE node
W20251213 08:58:10.959029 19496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:10.960238 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:10.960425 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:10.960552 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616290960541 us; error 0 us; skew 500 ppm
I20251213 08:58:10.961006 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:10.963253 18086 webserver.cc:492] Webserver started at http://127.17.169.129:38703/ using document root <none> and password file <none>
I20251213 08:58:10.963690 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:10.963851 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:10.964059 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:10.965133 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-0-root/instance:
uuid: "3560ca6c4983474ba5b1721e3b2dd9cf"
format_stamp: "Formatted at 2025-12-13 08:58:10 on dist-test-slave-s4vp"
I20251213 08:58:10.969707 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20251213 08:58:10.973012 19501 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:10.973958 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251213 08:58:10.974272 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-0-root
uuid: "3560ca6c4983474ba5b1721e3b2dd9cf"
format_stamp: "Formatted at 2025-12-13 08:58:10 on dist-test-slave-s4vp"
I20251213 08:58:10.974551 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:10.991684 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:10.992853 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:10.994338 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:58:10.996541 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:58:10.996739 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:10.996959 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:58:10.997107 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:11.038004 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.129:44921
I20251213 08:58:11.038105 19563 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.129:44921 every 8 connection(s)
I20251213 08:58:11.043576 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:11.050268 19568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:11.050776 19569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:11.055393 19572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:11.056106 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:11.056447 19564 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35755
I20251213 08:58:11.056823 19564 heartbeater.cc:461] Registering TS with master...
I20251213 08:58:11.057066 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:11.057281 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:11.057442 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616291057428 us; error 0 us; skew 500 ppm
I20251213 08:58:11.057528 19564 heartbeater.cc:507] Master 127.17.169.190:35755 requested a full tablet report, sending...
I20251213 08:58:11.057979 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:11.059669 19437 ts_manager.cc:194] Registered new tserver with Master: 3560ca6c4983474ba5b1721e3b2dd9cf (127.17.169.129:44921)
I20251213 08:58:11.060603 18086 webserver.cc:492] Webserver started at http://127.17.169.130:36877/ using document root <none> and password file <none>
I20251213 08:58:11.061069 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:11.061249 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:11.061553 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:11.061573 19437 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43960
I20251213 08:58:11.062839 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-1-root/instance:
uuid: "6d31f091c18f4c41b05c000086bc1cc9"
format_stamp: "Formatted at 2025-12-13 08:58:11 on dist-test-slave-s4vp"
I20251213 08:58:11.067070 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20251213 08:58:11.070314 19576 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:11.071192 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251213 08:58:11.071470 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-1-root
uuid: "6d31f091c18f4c41b05c000086bc1cc9"
format_stamp: "Formatted at 2025-12-13 08:58:11 on dist-test-slave-s4vp"
I20251213 08:58:11.071746 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:11.099589 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:11.100772 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:11.102234 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:58:11.104619 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:58:11.104909 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:11.105162 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:58:11.105315 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:11.147745 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.130:44367
I20251213 08:58:11.147854 19638 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.130:44367 every 8 connection(s)
I20251213 08:58:11.152325 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:11.158921 19642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:11.160225 19643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:11.163342 19645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:11.163646 19639 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35755
I20251213 08:58:11.164178 19639 heartbeater.cc:461] Registering TS with master...
I20251213 08:58:11.164834 19639 heartbeater.cc:507] Master 127.17.169.190:35755 requested a full tablet report, sending...
I20251213 08:58:11.165256 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:11.166049 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:11.166347 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:11.166525 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616291166507 us; error 0 us; skew 500 ppm
I20251213 08:58:11.166910 19437 ts_manager.cc:194] Registered new tserver with Master: 6d31f091c18f4c41b05c000086bc1cc9 (127.17.169.130:44367)
I20251213 08:58:11.167027 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:11.168358 19437 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43976
I20251213 08:58:11.169945 18086 webserver.cc:492] Webserver started at http://127.17.169.131:41399/ using document root <none> and password file <none>
I20251213 08:58:11.170447 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:11.170620 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:11.170869 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:11.171870 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-2-root/instance:
uuid: "321de0d602cb460e819b1ffa8cc539a6"
format_stamp: "Formatted at 2025-12-13 08:58:11 on dist-test-slave-s4vp"
I20251213 08:58:11.176429 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20251213 08:58:11.179466 19650 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:11.180147 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20251213 08:58:11.180408 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-2-root
uuid: "321de0d602cb460e819b1ffa8cc539a6"
format_stamp: "Formatted at 2025-12-13 08:58:11 on dist-test-slave-s4vp"
I20251213 08:58:11.180692 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_0.1765616224759991-18086-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:11.202600 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:11.203722 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:11.205127 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:58:11.207211 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:58:11.207413 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:11.207640 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:58:11.207796 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:11.247368 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.131:45243
I20251213 08:58:11.247507 19712 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.131:45243 every 8 connection(s)
I20251213 08:58:11.260248 19713 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35755
I20251213 08:58:11.260599 19713 heartbeater.cc:461] Registering TS with master...
I20251213 08:58:11.261270 19713 heartbeater.cc:507] Master 127.17.169.190:35755 requested a full tablet report, sending...
I20251213 08:58:11.263056 19437 ts_manager.cc:194] Registered new tserver with Master: 321de0d602cb460e819b1ffa8cc539a6 (127.17.169.131:45243)
I20251213 08:58:11.263725 18086 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013274389s
I20251213 08:58:11.264358 19437 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43982
I20251213 08:58:11.296312 18086 test_util.cc:276] Using random seed: -1814416350
I20251213 08:58:11.317807 19437 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44012:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251213 08:58:11.319700 19437 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251213 08:58:11.365061 19604 tablet_service.cc:1505] Processing CreateTablet for tablet 536bfdc98850485ca83206e51bd09964 (DEFAULT_TABLE table=test-workload [id=9e6e610538f64986af6214b21d5ab2a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:58:11.367158 19604 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536bfdc98850485ca83206e51bd09964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:58:11.380496 19529 tablet_service.cc:1505] Processing CreateTablet for tablet 536bfdc98850485ca83206e51bd09964 (DEFAULT_TABLE table=test-workload [id=9e6e610538f64986af6214b21d5ab2a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:58:11.381858 19529 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536bfdc98850485ca83206e51bd09964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:58:11.385427 19678 tablet_service.cc:1505] Processing CreateTablet for tablet 536bfdc98850485ca83206e51bd09964 (DEFAULT_TABLE table=test-workload [id=9e6e610538f64986af6214b21d5ab2a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:58:11.386685 19678 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 536bfdc98850485ca83206e51bd09964. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:58:11.390971 19734 tablet_bootstrap.cc:492] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9: Bootstrap starting.
I20251213 08:58:11.397516 19734 tablet_bootstrap.cc:654] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9: Neither blocks nor log segments found. Creating new log.
I20251213 08:58:11.399192 19735 tablet_bootstrap.cc:492] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf: Bootstrap starting.
I20251213 08:58:11.403132 19734 tablet_bootstrap.cc:492] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9: No bootstrap required, opened a new log
I20251213 08:58:11.403246 19737 tablet_bootstrap.cc:492] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6: Bootstrap starting.
I20251213 08:58:11.403786 19734 ts_tablet_manager.cc:1403] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.003s
I20251213 08:58:11.407109 19735 tablet_bootstrap.cc:654] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf: Neither blocks nor log segments found. Creating new log.
I20251213 08:58:11.407667 19734 raft_consensus.cc:359] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } }
I20251213 08:58:11.408463 19734 raft_consensus.cc:385] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:58:11.408664 19737 tablet_bootstrap.cc:654] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6: Neither blocks nor log segments found. Creating new log.
I20251213 08:58:11.408792 19734 raft_consensus.cc:740] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d31f091c18f4c41b05c000086bc1cc9, State: Initialized, Role: FOLLOWER
I20251213 08:58:11.409570 19734 consensus_queue.cc:260] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } }
I20251213 08:58:11.413178 19735 tablet_bootstrap.cc:492] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf: No bootstrap required, opened a new log
I20251213 08:58:11.413447 19639 heartbeater.cc:499] Master 127.17.169.190:35755 was elected leader, sending a full tablet report...
I20251213 08:58:11.413676 19735 ts_tablet_manager.cc:1403] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20251213 08:58:11.413908 19737 tablet_bootstrap.cc:492] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6: No bootstrap required, opened a new log
I20251213 08:58:11.414496 19737 ts_tablet_manager.cc:1403] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6: Time spent bootstrapping tablet: real 0.012s user 0.011s sys 0.000s
I20251213 08:58:11.414387 19734 ts_tablet_manager.cc:1434] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20251213 08:58:11.417966 19737 raft_consensus.cc:359] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } }
I20251213 08:58:11.417948 19735 raft_consensus.cc:359] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } }
I20251213 08:58:11.418749 19737 raft_consensus.cc:385] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:58:11.418924 19735 raft_consensus.cc:385] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:58:11.419075 19737 raft_consensus.cc:740] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 321de0d602cb460e819b1ffa8cc539a6, State: Initialized, Role: FOLLOWER
I20251213 08:58:11.419283 19735 raft_consensus.cc:740] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3560ca6c4983474ba5b1721e3b2dd9cf, State: Initialized, Role: FOLLOWER
I20251213 08:58:11.420001 19735 consensus_queue.cc:260] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } }
I20251213 08:58:11.419821 19737 consensus_queue.cc:260] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } }
I20251213 08:58:11.422214 19564 heartbeater.cc:499] Master 127.17.169.190:35755 was elected leader, sending a full tablet report...
I20251213 08:58:11.422876 19735 ts_tablet_manager.cc:1434] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20251213 08:58:11.426913 19713 heartbeater.cc:499] Master 127.17.169.190:35755 was elected leader, sending a full tablet report...
I20251213 08:58:11.427604 19737 ts_tablet_manager.cc:1434] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6: Time spent starting tablet: real 0.012s user 0.007s sys 0.003s
I20251213 08:58:11.444607 19741 raft_consensus.cc:493] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251213 08:58:11.445382 19741 raft_consensus.cc:515] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } }
I20251213 08:58:11.447728 19741 leader_election.cc:290] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d31f091c18f4c41b05c000086bc1cc9 (127.17.169.130:44367), 321de0d602cb460e819b1ffa8cc539a6 (127.17.169.131:45243)
I20251213 08:58:11.457914 19614 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "536bfdc98850485ca83206e51bd09964" candidate_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d31f091c18f4c41b05c000086bc1cc9" is_pre_election: true
I20251213 08:58:11.458803 19614 raft_consensus.cc:2468] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3560ca6c4983474ba5b1721e3b2dd9cf in term 0.
I20251213 08:58:11.458814 19688 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "536bfdc98850485ca83206e51bd09964" candidate_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "321de0d602cb460e819b1ffa8cc539a6" is_pre_election: true
I20251213 08:58:11.459573 19688 raft_consensus.cc:2468] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3560ca6c4983474ba5b1721e3b2dd9cf in term 0.
I20251213 08:58:11.460083 19503 leader_election.cc:304] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3560ca6c4983474ba5b1721e3b2dd9cf, 6d31f091c18f4c41b05c000086bc1cc9; no voters:
I20251213 08:58:11.460899 19741 raft_consensus.cc:2804] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251213 08:58:11.461241 19741 raft_consensus.cc:493] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251213 08:58:11.461548 19741 raft_consensus.cc:3060] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:58:11.466565 19741 raft_consensus.cc:515] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } }
I20251213 08:58:11.468134 19741 leader_election.cc:290] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [CANDIDATE]: Term 1 election: Requested vote from peers 6d31f091c18f4c41b05c000086bc1cc9 (127.17.169.130:44367), 321de0d602cb460e819b1ffa8cc539a6 (127.17.169.131:45243)
I20251213 08:58:11.468909 19614 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "536bfdc98850485ca83206e51bd09964" candidate_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d31f091c18f4c41b05c000086bc1cc9"
I20251213 08:58:11.469120 19688 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "536bfdc98850485ca83206e51bd09964" candidate_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "321de0d602cb460e819b1ffa8cc539a6"
I20251213 08:58:11.469355 19614 raft_consensus.cc:3060] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:58:11.469622 19688 raft_consensus.cc:3060] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:58:11.474628 19614 raft_consensus.cc:2468] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3560ca6c4983474ba5b1721e3b2dd9cf in term 1.
I20251213 08:58:11.474730 19688 raft_consensus.cc:2468] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3560ca6c4983474ba5b1721e3b2dd9cf in term 1.
I20251213 08:58:11.475613 19503 leader_election.cc:304] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3560ca6c4983474ba5b1721e3b2dd9cf, 6d31f091c18f4c41b05c000086bc1cc9; no voters:
I20251213 08:58:11.476272 19741 raft_consensus.cc:2804] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:58:11.477156 19741 raft_consensus.cc:697] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 1 LEADER]: Becoming Leader. State: Replica: 3560ca6c4983474ba5b1721e3b2dd9cf, State: Running, Role: LEADER
I20251213 08:58:11.477922 19741 consensus_queue.cc:237] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } }
I20251213 08:58:11.486129 19436 catalog_manager.cc:5654] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 3560ca6c4983474ba5b1721e3b2dd9cf (127.17.169.129). New cstate: current_term: 1 leader_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3560ca6c4983474ba5b1721e3b2dd9cf" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 44921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 } health_report { overall_health: UNKNOWN } } }
W20251213 08:58:11.501137 19714 tablet.cc:2378] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251213 08:58:11.508455 18086 test_util.cc:276] Using random seed: -1814204255
I20251213 08:58:11.534691 18086 test_workload.cc:405] TestWorkload: Skipping table creation because table test-workload already exists
I20251213 08:58:11.665180 19614 raft_consensus.cc:1275] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [term 1 FOLLOWER]: Refusing update from remote peer 3560ca6c4983474ba5b1721e3b2dd9cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:58:11.666579 19688 raft_consensus.cc:1275] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [term 1 FOLLOWER]: Refusing update from remote peer 3560ca6c4983474ba5b1721e3b2dd9cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:58:11.666512 19741 consensus_queue.cc:1048] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d31f091c18f4c41b05c000086bc1cc9" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 44367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251213 08:58:11.668179 19744 consensus_queue.cc:1048] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "321de0d602cb460e819b1ffa8cc539a6" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 45243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251213 08:58:12.047034 19426 maintenance_manager.cc:419] P 1ab2f27452ec4db4a09aea81623ee921: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.003577
I20251213 08:58:12.216045 19425 maintenance_manager.cc:643] P 1ab2f27452ec4db4a09aea81623ee921: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.168s user 0.037s sys 0.034s Metrics: {"bytes_written":6638,"cfile_init":1,"dirs.queue_time_us":20232,"dirs.run_cpu_time_us":319,"dirs.run_wall_time_us":1871,"drs_written":1,"lbm_read_time_us":339,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2953,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5,"thread_start_us":20491,"threads_started":1}
I20251213 08:58:12.218923 19426 maintenance_manager.cc:419] P 1ab2f27452ec4db4a09aea81623ee921: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 487 bytes on disk
I20251213 08:58:12.220773 19425 maintenance_manager.cc:643] P 1ab2f27452ec4db4a09aea81623ee921: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":337,"lbm_reads_lt_1ms":4}
I20251213 08:58:12.513847 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.242942
I20251213 08:58:12.550765 19565 maintenance_manager.cc:419] P 3560ca6c4983474ba5b1721e3b2dd9cf: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.242942
I20251213 08:58:12.655901 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.242942
I20251213 08:58:13.269466 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.755s user 0.454s sys 0.020s Metrics: {"bytes_written":43399,"cfile_init":1,"dirs.queue_time_us":25877,"dirs.run_cpu_time_us":338,"dirs.run_wall_time_us":3835,"drs_written":1,"lbm_read_time_us":275,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2611,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1500,"thread_start_us":25529,"threads_started":1}
I20251213 08:58:13.272701 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 13032 bytes on disk
I20251213 08:58:13.275290 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":289,"lbm_reads_lt_1ms":4}
I20251213 08:58:13.409396 19506 maintenance_manager.cc:643] P 3560ca6c4983474ba5b1721e3b2dd9cf: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.858s user 0.436s sys 0.032s Metrics: {"bytes_written":44695,"cfile_init":1,"dirs.queue_time_us":6821,"dirs.run_cpu_time_us":307,"dirs.run_wall_time_us":5423,"drs_written":1,"lbm_read_time_us":256,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2657,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1550,"thread_start_us":7038,"threads_started":1}
I20251213 08:58:13.424417 19565 maintenance_manager.cc:419] P 3560ca6c4983474ba5b1721e3b2dd9cf: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 13455 bytes on disk
I20251213 08:58:13.426219 19506 maintenance_manager.cc:643] P 3560ca6c4983474ba5b1721e3b2dd9cf: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.001s Metrics: {"cfile_init":1,"lbm_read_time_us":277,"lbm_reads_lt_1ms":4}
I20251213 08:58:13.689940 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.033s user 0.496s sys 0.028s Metrics: {"bytes_written":48655,"cfile_init":1,"dirs.queue_time_us":41856,"dirs.run_cpu_time_us":312,"dirs.run_wall_time_us":10514,"drs_written":1,"lbm_read_time_us":302,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2766,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1700,"spinlock_wait_cycles":585472,"thread_start_us":47465,"threads_started":1}
I20251213 08:58:13.692937 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 14747 bytes on disk
I20251213 08:58:13.694666 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":297,"lbm_reads_lt_1ms":4}
I20251213 08:58:14.283526 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.486106
I20251213 08:58:14.454030 19565 maintenance_manager.cc:419] P 3560ca6c4983474ba5b1721e3b2dd9cf: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.486106
I20251213 08:58:14.700435 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.486106
I20251213 08:58:15.582406 19506 maintenance_manager.cc:643] P 3560ca6c4983474ba5b1721e3b2dd9cf: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.128s user 0.850s sys 0.038s Metrics: {"bytes_written":91215,"cfile_init":1,"dirs.queue_time_us":37792,"dirs.run_cpu_time_us":321,"dirs.run_wall_time_us":6570,"drs_written":1,"lbm_read_time_us":280,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3428,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3200,"spinlock_wait_cycles":215936,"thread_start_us":37516,"threads_started":1}
I20251213 08:58:15.587874 19565 maintenance_manager.cc:419] P 3560ca6c4983474ba5b1721e3b2dd9cf: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 28780 bytes on disk
I20251213 08:58:15.589761 19506 maintenance_manager.cc:643] P 3560ca6c4983474ba5b1721e3b2dd9cf: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":296,"lbm_reads_lt_1ms":4}
I20251213 08:58:15.592357 19565 maintenance_manager.cc:419] P 3560ca6c4983474ba5b1721e3b2dd9cf: Scheduling CompactRowSetsOp(536bfdc98850485ca83206e51bd09964): perf score=1.014719
W20251213 08:58:15.823882 19747 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.17.169.129:44921, user_credentials={real_user=slave}} blocked reactor thread for 46700.2us
I20251213 08:58:15.877115 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.593s user 0.765s sys 0.024s Metrics: {"bytes_written":83265,"cfile_init":1,"dirs.queue_time_us":6897,"dirs.run_cpu_time_us":383,"dirs.run_wall_time_us":7480,"drs_written":1,"lbm_read_time_us":286,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3290,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2900,"thread_start_us":7145,"threads_started":1}
I20251213 08:58:15.886596 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 26021 bytes on disk
I20251213 08:58:15.888821 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":363,"lbm_reads_lt_1ms":4}
I20251213 08:58:15.891880 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling CompactRowSetsOp(536bfdc98850485ca83206e51bd09964): perf score=1.014419
I20251213 08:58:16.667038 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.966s user 0.953s sys 0.106s Metrics: {"bytes_written":94825,"cfile_init":1,"dirs.queue_time_us":149933,"dirs.run_cpu_time_us":375,"dirs.run_wall_time_us":6778,"drs_written":1,"lbm_read_time_us":277,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3423,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3350,"spinlock_wait_cycles":228096,"thread_start_us":149544,"threads_started":1}
I20251213 08:58:16.671766 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 30203 bytes on disk
I20251213 08:58:16.701988 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":274,"lbm_reads_lt_1ms":4}
I20251213 08:58:16.704984 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling CompactRowSetsOp(536bfdc98850485ca83206e51bd09964): perf score=1.014747
I20251213 08:58:17.200913 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: CompactRowSetsOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.308s user 0.884s sys 0.043s Metrics: {"bytes_written":126485,"cfile_cache_hit":10,"cfile_cache_hit_bytes":80535,"cfile_cache_miss":28,"cfile_cache_miss_bytes":224615,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":9954,"dirs.run_cpu_time_us":1697,"dirs.run_wall_time_us":12518,"drs_written":1,"lbm_read_time_us":9487,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":47,"lbm_write_time_us":3900,"lbm_writes_lt_1ms":26,"mutex_wait_us":74,"num_input_rowsets":2,"peak_mem_usage":69401,"rows_written":4400,"spinlock_wait_cycles":47104,"thread_start_us":16462,"threads_started":4}
I20251213 08:58:17.204560 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.486106
I20251213 08:58:17.601293 19506 maintenance_manager.cc:643] P 3560ca6c4983474ba5b1721e3b2dd9cf: CompactRowSetsOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 2.008s user 0.953s sys 0.047s Metrics: {"bytes_written":135697,"cfile_cache_hit":12,"cfile_cache_hit_bytes":97699,"cfile_cache_miss":26,"cfile_cache_miss_bytes":232307,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":7593,"dirs.run_cpu_time_us":357,"dirs.run_wall_time_us":11814,"drs_written":1,"lbm_read_time_us":45412,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":44,"lbm_write_time_us":4177,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":73041,"rows_written":4750,"thread_start_us":7206,"threads_started":1}
I20251213 08:58:17.605136 19565 maintenance_manager.cc:419] P 3560ca6c4983474ba5b1721e3b2dd9cf: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.486106
W20251213 08:58:17.947214 19747 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerService.Write -> {remote=127.17.169.129:44921, user_credentials={real_user=slave}} blocked reactor thread for 51448.4us
I20251213 08:58:18.728068 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.523s user 0.879s sys 0.032s Metrics: {"bytes_written":92437,"cfile_init":1,"dirs.queue_time_us":43483,"dirs.run_cpu_time_us":309,"dirs.run_wall_time_us":9784,"drs_written":1,"lbm_read_time_us":421,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3567,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3250,"thread_start_us":37746,"threads_started":1}
I20251213 08:58:18.731528 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 68921 bytes on disk
I20251213 08:58:18.733762 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":559,"lbm_reads_lt_1ms":8}
I20251213 08:58:18.736308 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling CompactRowSetsOp(536bfdc98850485ca83206e51bd09964): perf score=1.014771
I20251213 08:58:18.943131 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: CompactRowSetsOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 2.237s user 0.998s sys 0.030s Metrics: {"bytes_written":143732,"cfile_cache_hit":12,"cfile_cache_hit_bytes":104216,"cfile_cache_miss":26,"cfile_cache_miss_bytes":246360,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":8745,"dirs.run_cpu_time_us":326,"dirs.run_wall_time_us":72248,"drs_written":1,"lbm_read_time_us":18009,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":44,"lbm_write_time_us":3960,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":77466,"rows_written":5050,"thread_start_us":8955,"threads_started":1}
I20251213 08:58:18.946388 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 46330 bytes on disk
I20251213 08:58:18.957255 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":278,"lbm_reads_lt_1ms":4}
I20251213 08:58:18.959380 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.486106
I20251213 08:58:19.342303 18086 stop_tablet-itest.cc:98] Stopping T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf
I20251213 08:58:19.342845 18086 stop_tablet-itest.cc:261] Waiting for writes to settle...
W20251213 08:58:19.347448 19805 mvcc.cc:118] aborting op with timestamp 7231964362089058304 in state 0; MVCC is closed
W20251213 08:58:19.348220 19805 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:58738: Illegal state: Not Applying op; the tablet is stopped
I20251213 08:58:19.403297 19506 maintenance_manager.cc:643] P 3560ca6c4983474ba5b1721e3b2dd9cf: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.797s user 0.859s sys 0.042s Metrics: {"bytes_written":93845,"cfile_init":1,"dirs.queue_time_us":6852,"dirs.run_cpu_time_us":325,"dirs.run_wall_time_us":8150,"drs_written":1,"lbm_read_time_us":285,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3779,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3300,"spinlock_wait_cycles":8448,"thread_start_us":7058,"threads_started":1}
W20251213 08:58:19.410739 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.413172 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.416695 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.442441 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.443427 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.446203 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.446666 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.453111 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.469980 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.474864 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.477012 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.480015 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.488278 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.488936 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.491559 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.503347 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.509595 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.519858 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.522274 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.523994 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.529983 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.534569 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.536334 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.543560 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.559890 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.567283 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.575076 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.575829 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.579001 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.585214 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.592392 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.595106 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.619561 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.620020 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.634676 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.640877 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.641498 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.644793 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.657604 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.663759 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.691396 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.692953 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.707039 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.709659 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.716400 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.714538 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.729779 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.733212 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.773154 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.773240 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.783263 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.785564 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.801661 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.801771 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.812263 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.812930 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.864400 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.864831 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.868703 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.875389 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.896559 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.897076 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.900547 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.905054 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.962885 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.965289 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.968150 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.971441 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.996769 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:19.999123 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.002182 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.002810 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.074828 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.075274 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.077260 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.081456 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.111940 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.112493 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.115581 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.116824 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.191335 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.195156 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.196732 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.199616 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
I20251213 08:58:20.214200 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.253s user 0.893s sys 0.024s Metrics: {"bytes_written":129134,"cfile_init":1,"dirs.queue_time_us":856,"dirs.run_cpu_time_us":348,"dirs.run_wall_time_us":1937,"drs_written":1,"lbm_read_time_us":169,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3181,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":4500,"spinlock_wait_cycles":232704,"thread_start_us":522,"threads_started":1}
I20251213 08:58:20.217355 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 40753 bytes on disk
I20251213 08:58:20.218858 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":199,"lbm_reads_lt_1ms":4}
I20251213 08:58:20.221104 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling CompactRowSetsOp(536bfdc98850485ca83206e51bd09964): perf score=1.014721
W20251213 08:58:20.227911 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.233728 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.237751 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.238308 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
I20251213 08:58:20.285431 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: CompactRowSetsOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.543s user 1.158s sys 0.027s Metrics: {"bytes_written":220781,"cfile_cache_hit":10,"cfile_cache_hit_bytes":140179,"cfile_cache_miss":30,"cfile_cache_miss_bytes":392753,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":5191,"dirs.run_cpu_time_us":1764,"dirs.run_wall_time_us":7228,"drs_written":1,"lbm_read_time_us":5726,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":49,"lbm_write_time_us":4551,"lbm_writes_lt_1ms":30,"mutex_wait_us":5,"num_input_rowsets":2,"peak_mem_usage":116521,"rows_written":7650,"thread_start_us":4713,"threads_started":4}
I20251213 08:58:20.287972 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 70988 bytes on disk
I20251213 08:58:20.289222 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":212,"lbm_reads_lt_1ms":4}
I20251213 08:58:20.290632 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.486106
W20251213 08:58:20.309052 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.318640 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.321825 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.324160 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.349570 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.358038 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.364023 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.365921 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.440227 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.449466 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.452302 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.457109 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.484807 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.493767 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.499336 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.501943 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.577363 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.590399 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.596309 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.596791 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.623564 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.639102 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.642601 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.645226 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.722736 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
I20251213 08:58:20.737175 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.446s user 0.438s sys 0.004s Metrics: {"bytes_written":70539,"cfile_init":1,"dirs.queue_time_us":122,"dirs.run_cpu_time_us":287,"dirs.run_wall_time_us":1462,"drs_written":1,"lbm_read_time_us":174,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2649,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2400}
I20251213 08:58:20.740065 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling CompactRowSetsOp(536bfdc98850485ca83206e51bd09964): perf score=1.013826
W20251213 08:58:20.740219 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.744171 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.746225 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.773367 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.792913 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.795764 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.796365 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.879432 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.904546 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.906060 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.908910 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.932945 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.959834 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.961028 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:20.964618 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.042562 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.070878 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.075078 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.079141 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.100338 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.126013 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.131353 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.135428 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.221060 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.244298 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.253258 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.255924 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.281020 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.301857 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.314446 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.314711 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.407579 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.427374 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.437311 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.437738 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
I20251213 08:58:21.464670 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: CompactRowSetsOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.243s user 1.212s sys 0.028s Metrics: {"bytes_written":275071,"cfile_cache_hit":10,"cfile_cache_hit_bytes":175955,"cfile_cache_miss":34,"cfile_cache_miss_bytes":492945,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":3267,"dirs.run_cpu_time_us":829,"dirs.run_wall_time_us":3769,"drs_written":1,"lbm_read_time_us":6403,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":52,"lbm_write_time_us":4685,"lbm_writes_lt_1ms":32,"mutex_wait_us":125,"num_input_rowsets":2,"peak_mem_usage":145487,"rows_written":9550,"spinlock_wait_cycles":25984,"thread_start_us":3072,"threads_started":5,"wal-append.queue_time_us":999}
I20251213 08:58:21.467553 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling FlushMRSOp(536bfdc98850485ca83206e51bd09964): perf score=0.060569
W20251213 08:58:21.472254 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.487823 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.498355 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.502219 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
I20251213 08:58:21.566082 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: FlushMRSOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.098s user 0.094s sys 0.000s Metrics: {"bytes_written":17724,"cfile_init":1,"dirs.queue_time_us":105,"dirs.run_cpu_time_us":271,"dirs.run_wall_time_us":853,"drs_written":1,"lbm_read_time_us":222,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1664,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":500}
I20251213 08:58:21.569803 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 93940 bytes on disk
I20251213 08:58:21.572331 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.002s user 0.000s sys 0.000s Metrics: {"cfile_init":2,"lbm_read_time_us":537,"lbm_reads_lt_1ms":8}
I20251213 08:58:21.574609 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling CompactRowSetsOp(536bfdc98850485ca83206e51bd09964): perf score=1.007447
W20251213 08:58:21.601665 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.618536 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.635803 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.638505 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.667824 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.683362 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.700783 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.701333 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.803130 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.823061 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.837756 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.841195 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.870361 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.892997 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.906520 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:21.907732 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.012012 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
I20251213 08:58:22.016342 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: CompactRowSetsOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.275s user 1.246s sys 0.028s Metrics: {"bytes_written":288198,"cfile_cache_hit":10,"cfile_cache_hit_bytes":187275,"cfile_cache_miss":34,"cfile_cache_miss_bytes":520581,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":3228,"dirs.run_cpu_time_us":1190,"dirs.run_wall_time_us":5551,"drs_written":1,"lbm_read_time_us":5386,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":57,"lbm_write_time_us":4482,"lbm_writes_lt_1ms":32,"mutex_wait_us":134,"num_input_rowsets":2,"peak_mem_usage":153112,"rows_written":10050,"spinlock_wait_cycles":5938304,"thread_start_us":2379,"threads_started":4,"wal-append.queue_time_us":962}
I20251213 08:58:22.018915 19714 maintenance_manager.cc:419] P 321de0d602cb460e819b1ffa8cc539a6: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 94228 bytes on disk
I20251213 08:58:22.020256 19655 maintenance_manager.cc:643] P 321de0d602cb460e819b1ffa8cc539a6: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":189,"lbm_reads_lt_1ms":4}
W20251213 08:58:22.033928 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.050987 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.051179 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.081279 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.104048 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.121690 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.123903 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.231952 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.253512 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.268234 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.271250 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.306392 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.328074 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.345105 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.347525 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.458648 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.478107 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.496798 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.500974 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.534008 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.557410 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.574882 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.579071 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.694631 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.715502 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.733960 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.739173 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
I20251213 08:58:22.757388 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: CompactRowSetsOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 1.182s user 1.165s sys 0.016s Metrics: {"bytes_written":288198,"cfile_cache_hit":10,"cfile_cache_hit_bytes":189175,"cfile_cache_miss":34,"cfile_cache_miss_bytes":525057,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":2930,"dirs.run_cpu_time_us":1899,"dirs.run_wall_time_us":7026,"drs_written":1,"lbm_read_time_us":5484,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":53,"lbm_write_time_us":4171,"lbm_writes_lt_1ms":32,"mutex_wait_us":241,"num_input_rowsets":2,"peak_mem_usage":154900,"rows_written":10050,"spinlock_wait_cycles":1792,"thread_start_us":3147,"threads_started":6,"wal-append.queue_time_us":718}
I20251213 08:58:22.759672 19640 maintenance_manager.cc:419] P 6d31f091c18f4c41b05c000086bc1cc9: Scheduling UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964): 94228 bytes on disk
I20251213 08:58:22.760896 19581 maintenance_manager.cc:643] P 6d31f091c18f4c41b05c000086bc1cc9: UndoDeltaBlockGCOp(536bfdc98850485ca83206e51bd09964) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":201,"lbm_reads_lt_1ms":4}
W20251213 08:58:22.776746 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.796259 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.814079 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.818032 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.939795 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.962417 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.978828 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:22.982882 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.023571 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.043161 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.062950 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.067143 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.193578 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.212850 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.232210 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.239195 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.279707 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.297569 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.320020 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.323231 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.459664 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.473311 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.496104 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.497649 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.550382 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.562849 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.584390 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.585536 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.734055 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.749871 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.767660 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.770815 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.828924 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.844085 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.859277 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:23.865264 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.022040 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.031358 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.053573 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.053573 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.117277 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.130198 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.148186 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.148497 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.319304 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.327854 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.348676 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.348678 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.416270 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.426256 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.448889 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.448949 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.617436 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.627501 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.651930 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.654358 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.719208 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.728488 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.753518 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.757050 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.927980 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.939960 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.963609 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:24.963743 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.032488 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.044472 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.068217 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.070375 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.246083 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.260253 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.284684 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.287995 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.352873 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.369464 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.395510 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.395625 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.578332 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.588608 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.613760 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.617245 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.691531 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.696977 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.725235 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.729444 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.954835 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.954835 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.955857 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:25.957527 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.069550 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.070307 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.072571 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.072649 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.305254 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.306981 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.308708 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.310138 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.421532 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.422735 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.425801 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.426925 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.662086 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.662142 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.665037 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.665318 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.781702 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.782013 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.784484 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:26.785565 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.028968 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.030805 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.032971 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.033290 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.153326 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.153496 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.154861 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.155383 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.403911 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.407244 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.407429 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.407521 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.531157 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.531198 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.532580 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.534440 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.790339 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.790393 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.792624 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.794750 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.917490 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.918749 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.920354 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:27.923496 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.183701 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.183737 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.184212 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.187862 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.314903 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.314903 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.316323 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.316972 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.584951 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.585165 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.587313 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.587363 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.719856 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.719875 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.723641 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.723716 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.998186 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.998319 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:28.998273 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.002367 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.136449 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.136453 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.137908 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.138094 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
I20251213 08:58:29.343632 18086 stop_tablet-itest.cc:271] Waiting to ensure no more writes come in...
W20251213 08:58:29.419497 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.419996 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.420228 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.420753 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.560595 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.560721 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.562865 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.563052 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.852242 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.853315 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.853444 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.853720 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.995219 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.995231 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.996858 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:29.999131 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.289718 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.291550 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.292021 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.293524 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.436447 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.436750 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.437959 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.439949 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.737210 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.737434 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.739431 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.739756 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.891093 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.891539 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.892086 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:30.891402 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.197788 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.197800 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.198047 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.198571 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.351104 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.353590 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.354095 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.354525 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.664626 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.666509 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.667662 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.669402 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.820842 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.822139 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.824234 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:31.824997 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.137136 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.138710 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.140475 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.140789 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.295871 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.296742 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.299281 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.300648 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.617692 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.619354 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.622293 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.622876 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.779299 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.779336 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.782390 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:32.783111 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.110379 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.110567 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.112680 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.117076 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.276983 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.277794 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.279703 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.280004 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.613207 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.613668 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.614068 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.616987 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.779623 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.780997 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.782323 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:33.783811 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.117563 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.123160 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.123160 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.125032 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.289860 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.292148 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.295490 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.297154 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
I20251213 08:58:34.344419 18086 stop_tablet-itest.cc:274] Success! No more writes came in!
W20251213 08:58:34.638559 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.638592 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.640347 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.642252 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.812106 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.815363 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.815812 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:34.815989 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.165720 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.167413 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.170480 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.171249 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.342391 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.344808 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.346315 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.347677 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.704260 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.704452 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.704733 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.705104 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.888811 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.888811 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.888813 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:35.889931 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.257148 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.259729 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.261526 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.261819 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.441170 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.442107 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.442548 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.444200 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.812016 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.813921 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.813921 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.817662 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.997128 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.998313 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:36.998385 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.003809 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.375681 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.375715 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.375726 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.380971 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.565171 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.565335 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.566962 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.569432 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.948458 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.948848 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.950567 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:37.952625 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.140702 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.140740 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.143448 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.145699 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.529093 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.533022 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.533257 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.535133 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.724267 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.726794 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.729183 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:38.730998 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:39.123586 19592 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:39.123991 19594 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:39.123569 19593 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:39.124799 19591 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:44890: Illegal state: replica 6d31f091c18f4c41b05c000086bc1cc9 is not leader of this config: current role FOLLOWER
W20251213 08:58:39.322893 19667 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:39.322801 19668 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:39.323550 19666 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:39.323207 19665 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55832: Illegal state: replica 321de0d602cb460e819b1ffa8cc539a6 is not leader of this config: current role FOLLOWER
W20251213 08:58:39.518813 19746 batcher.cc:441] Timed out: Failed to write batch of 50 ops to tablet 536bfdc98850485ca83206e51bd09964 after 193 attempt(s): Failed to write to server: (no server available): Write(tablet: 536bfdc98850485ca83206e51bd09964, num_ops: 50, num_attempts: 193) passed its deadline: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:39.530728 18086 tablet_server.cc:178] TabletServer@127.17.169.129:0 shutting down...
I20251213 08:58:39.567153 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:58:39.568498 18086 tablet_replica.cc:333] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf: stopping tablet replica
I20251213 08:58:39.569751 18086 raft_consensus.cc:2243] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:58:39.570942 18086 raft_consensus.cc:2272] T 536bfdc98850485ca83206e51bd09964 P 3560ca6c4983474ba5b1721e3b2dd9cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:39.598843 18086 tablet_server.cc:195] TabletServer@127.17.169.129:0 shutdown complete.
I20251213 08:58:39.617231 18086 tablet_server.cc:178] TabletServer@127.17.169.130:0 shutting down...
I20251213 08:58:39.647567 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:58:39.648403 18086 tablet_replica.cc:333] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9: stopping tablet replica
I20251213 08:58:39.649108 18086 raft_consensus.cc:2243] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:58:39.649829 18086 raft_consensus.cc:2272] T 536bfdc98850485ca83206e51bd09964 P 6d31f091c18f4c41b05c000086bc1cc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:39.673236 18086 tablet_server.cc:195] TabletServer@127.17.169.130:0 shutdown complete.
I20251213 08:58:39.691599 18086 tablet_server.cc:178] TabletServer@127.17.169.131:0 shutting down...
I20251213 08:58:39.720891 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:58:39.721737 18086 tablet_replica.cc:333] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6: stopping tablet replica
I20251213 08:58:39.722432 18086 raft_consensus.cc:2243] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:58:39.723168 18086 raft_consensus.cc:2272] T 536bfdc98850485ca83206e51bd09964 P 321de0d602cb460e819b1ffa8cc539a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:39.747460 18086 tablet_server.cc:195] TabletServer@127.17.169.131:0 shutdown complete.
I20251213 08:58:39.769797 18086 master.cc:561] Master@127.17.169.190:35755 shutting down...
I20251213 08:58:39.800043 18086 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:58:39.800791 18086 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:39.801110 18086 tablet_replica.cc:333] T 00000000000000000000000000000000 P 1ab2f27452ec4db4a09aea81623ee921: stopping tablet replica
I20251213 08:58:39.825403 18086 master.cc:583] Master@127.17.169.190:35755 shutdown complete.
[ OK ] StopTablets/StopTabletITest.TestStoppedTabletsDontWrite/0 (29122 ms)
[ RUN ] StopTablets/StopTabletITest.TestStoppedTabletsDontWrite/1
I20251213 08:58:39.871973 18086 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.17.169.190:35231
I20251213 08:58:39.873289 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:39.879995 19862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:39.880159 19863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:39.880446 19865 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:39.883173 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:39.884310 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:39.884526 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:39.884646 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616319884634 us; error 0 us; skew 500 ppm
I20251213 08:58:39.885135 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:39.888293 18086 webserver.cc:492] Webserver started at http://127.17.169.190:44559/ using document root <none> and password file <none>
I20251213 08:58:39.888813 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:39.888989 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:39.889214 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:39.890588 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/master-0-root/instance:
uuid: "149552c1ca614c8ab58059cb417e913b"
format_stamp: "Formatted at 2025-12-13 08:58:39 on dist-test-slave-s4vp"
I20251213 08:58:39.895594 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20251213 08:58:39.899376 19870 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:39.900247 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20251213 08:58:39.900534 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/master-0-root
uuid: "149552c1ca614c8ab58059cb417e913b"
format_stamp: "Formatted at 2025-12-13 08:58:39 on dist-test-slave-s4vp"
I20251213 08:58:39.900815 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:39.934435 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:39.935917 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:39.981031 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.190:35231
I20251213 08:58:39.981138 19921 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.190:35231 every 8 connection(s)
I20251213 08:58:39.985046 19922 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:58:39.995998 19922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b: Bootstrap starting.
I20251213 08:58:40.000917 19922 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b: Neither blocks nor log segments found. Creating new log.
I20251213 08:58:40.005733 19922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b: No bootstrap required, opened a new log
I20251213 08:58:40.007932 19922 raft_consensus.cc:359] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "149552c1ca614c8ab58059cb417e913b" member_type: VOTER }
I20251213 08:58:40.008376 19922 raft_consensus.cc:385] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:58:40.008623 19922 raft_consensus.cc:740] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149552c1ca614c8ab58059cb417e913b, State: Initialized, Role: FOLLOWER
I20251213 08:58:40.009398 19922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "149552c1ca614c8ab58059cb417e913b" member_type: VOTER }
I20251213 08:58:40.009864 19922 raft_consensus.cc:399] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:58:40.010123 19922 raft_consensus.cc:493] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:58:40.010434 19922 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:58:40.015112 19922 raft_consensus.cc:515] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "149552c1ca614c8ab58059cb417e913b" member_type: VOTER }
I20251213 08:58:40.015647 19922 leader_election.cc:304] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 149552c1ca614c8ab58059cb417e913b; no voters:
I20251213 08:58:40.017149 19922 leader_election.cc:290] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:58:40.017617 19925 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:58:40.019248 19925 raft_consensus.cc:697] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 1 LEADER]: Becoming Leader. State: Replica: 149552c1ca614c8ab58059cb417e913b, State: Running, Role: LEADER
I20251213 08:58:40.020049 19925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "149552c1ca614c8ab58059cb417e913b" member_type: VOTER }
I20251213 08:58:40.020536 19922 sys_catalog.cc:565] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [sys.catalog]: configured and running, proceeding with master startup.
I20251213 08:58:40.025193 19926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "149552c1ca614c8ab58059cb417e913b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "149552c1ca614c8ab58059cb417e913b" member_type: VOTER } }
I20251213 08:58:40.025913 19926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [sys.catalog]: This master's current role is: LEADER
I20251213 08:58:40.025177 19927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 149552c1ca614c8ab58059cb417e913b. Latest consensus state: current_term: 1 leader_uuid: "149552c1ca614c8ab58059cb417e913b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "149552c1ca614c8ab58059cb417e913b" member_type: VOTER } }
I20251213 08:58:40.026971 19927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [sys.catalog]: This master's current role is: LEADER
I20251213 08:58:40.032193 19932 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251213 08:58:40.037492 19932 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251213 08:58:40.040872 18086 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251213 08:58:40.048472 19932 catalog_manager.cc:1357] Generated new cluster ID: 97eebecd8847454da0b65d24b0346060
I20251213 08:58:40.048820 19932 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251213 08:58:40.068255 19932 catalog_manager.cc:1380] Generated new certificate authority record
I20251213 08:58:40.069643 19932 catalog_manager.cc:1514] Loading token signing keys...
I20251213 08:58:40.087067 19932 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b: Generated new TSK 0
I20251213 08:58:40.087702 19932 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251213 08:58:40.113052 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:40.119267 19943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:40.120384 19944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:40.121758 19946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:40.122478 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:40.123286 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:40.123474 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:40.123598 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616320123587 us; error 0 us; skew 500 ppm
I20251213 08:58:40.124047 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:40.126398 18086 webserver.cc:492] Webserver started at http://127.17.169.129:38175/ using document root <none> and password file <none>
I20251213 08:58:40.126833 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:40.126984 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:40.127192 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:40.128352 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-0-root/instance:
uuid: "95695f48d06342a2b8f760e3001d3790"
format_stamp: "Formatted at 2025-12-13 08:58:40 on dist-test-slave-s4vp"
I20251213 08:58:40.132792 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20251213 08:58:40.136229 19951 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:40.137107 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251213 08:58:40.137426 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-0-root
uuid: "95695f48d06342a2b8f760e3001d3790"
format_stamp: "Formatted at 2025-12-13 08:58:40 on dist-test-slave-s4vp"
I20251213 08:58:40.137717 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:40.169296 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:40.170569 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:40.172021 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:58:40.174286 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:58:40.174479 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:40.174708 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:58:40.174863 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:40.219859 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.129:43929
I20251213 08:58:40.219969 20013 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.129:43929 every 8 connection(s)
I20251213 08:58:40.224634 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:40.233605 20019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:40.234848 20018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:40.235689 20021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:40.236765 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:40.238083 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:40.238372 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:40.238588 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616320238569 us; error 0 us; skew 500 ppm
I20251213 08:58:40.239056 20014 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35231
I20251213 08:58:40.239289 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:40.239429 20014 heartbeater.cc:461] Registering TS with master...
I20251213 08:58:40.240195 20014 heartbeater.cc:507] Master 127.17.169.190:35231 requested a full tablet report, sending...
I20251213 08:58:40.242106 18086 webserver.cc:492] Webserver started at http://127.17.169.130:38285/ using document root <none> and password file <none>
I20251213 08:58:40.242579 19887 ts_manager.cc:194] Registered new tserver with Master: 95695f48d06342a2b8f760e3001d3790 (127.17.169.129:43929)
I20251213 08:58:40.242816 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:40.243099 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:40.243429 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:40.244875 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-1-root/instance:
uuid: "0876bd2980b4442fb646de497ee70b79"
format_stamp: "Formatted at 2025-12-13 08:58:40 on dist-test-slave-s4vp"
I20251213 08:58:40.245102 19887 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37168
I20251213 08:58:40.249768 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20251213 08:58:40.253528 20026 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:40.254498 18086 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20251213 08:58:40.254779 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-1-root
uuid: "0876bd2980b4442fb646de497ee70b79"
format_stamp: "Formatted at 2025-12-13 08:58:40 on dist-test-slave-s4vp"
I20251213 08:58:40.255029 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:40.274001 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:40.275190 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:40.276647 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:58:40.278760 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:58:40.278964 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:40.279186 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:58:40.279345 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:40.323233 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.130:35039
I20251213 08:58:40.323350 20088 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.130:35039 every 8 connection(s)
I20251213 08:58:40.328262 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:40.336203 20092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:40.336467 20093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:40.340155 20095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:40.341013 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:40.342129 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:40.342404 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:40.342361 20089 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35231
I20251213 08:58:40.342754 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616320342735 us; error 0 us; skew 500 ppm
I20251213 08:58:40.342963 20089 heartbeater.cc:461] Registering TS with master...
I20251213 08:58:40.343505 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:40.343878 20089 heartbeater.cc:507] Master 127.17.169.190:35231 requested a full tablet report, sending...
I20251213 08:58:40.346294 19887 ts_manager.cc:194] Registered new tserver with Master: 0876bd2980b4442fb646de497ee70b79 (127.17.169.130:35039)
I20251213 08:58:40.346486 18086 webserver.cc:492] Webserver started at http://127.17.169.131:38415/ using document root <none> and password file <none>
I20251213 08:58:40.347158 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:40.347406 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:40.347728 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:40.348155 19887 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37176
I20251213 08:58:40.349303 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-2-root/instance:
uuid: "b3421c084c4947d8a570edc0c7d913ad"
format_stamp: "Formatted at 2025-12-13 08:58:40 on dist-test-slave-s4vp"
I20251213 08:58:40.353710 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20251213 08:58:40.357128 20100 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:40.357877 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251213 08:58:40.358143 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-2-root
uuid: "b3421c084c4947d8a570edc0c7d913ad"
format_stamp: "Formatted at 2025-12-13 08:58:40 on dist-test-slave-s4vp"
I20251213 08:58:40.358439 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestStoppedTabletsDontWrite_1.1765616224759991-18086-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:40.371471 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:40.372587 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:40.374033 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:58:40.376236 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:58:40.376442 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:40.376678 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:58:40.376834 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:40.418330 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.131:43485
I20251213 08:58:40.418462 20162 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.131:43485 every 8 connection(s)
I20251213 08:58:40.431878 20163 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35231
I20251213 08:58:40.432240 20163 heartbeater.cc:461] Registering TS with master...
I20251213 08:58:40.432905 20163 heartbeater.cc:507] Master 127.17.169.190:35231 requested a full tablet report, sending...
I20251213 08:58:40.434755 19887 ts_manager.cc:194] Registered new tserver with Master: b3421c084c4947d8a570edc0c7d913ad (127.17.169.131:43485)
I20251213 08:58:40.435073 18086 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013612475s
I20251213 08:58:40.436301 19887 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37184
I20251213 08:58:40.467573 18086 test_util.cc:276] Using random seed: -1785245090
I20251213 08:58:40.488583 19887 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37204:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251213 08:58:40.490543 19887 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251213 08:58:40.526458 20128 tablet_service.cc:1505] Processing CreateTablet for tablet c622e37af7354d1f9b556f7ccb4c2ee1 (DEFAULT_TABLE table=test-workload [id=0142d22c2107438d983fca9555b5f6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:58:40.527812 20128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c622e37af7354d1f9b556f7ccb4c2ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:58:40.532639 19979 tablet_service.cc:1505] Processing CreateTablet for tablet c622e37af7354d1f9b556f7ccb4c2ee1 (DEFAULT_TABLE table=test-workload [id=0142d22c2107438d983fca9555b5f6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:58:40.533090 20054 tablet_service.cc:1505] Processing CreateTablet for tablet c622e37af7354d1f9b556f7ccb4c2ee1 (DEFAULT_TABLE table=test-workload [id=0142d22c2107438d983fca9555b5f6e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:58:40.533748 19979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c622e37af7354d1f9b556f7ccb4c2ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:58:40.534399 20054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c622e37af7354d1f9b556f7ccb4c2ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:58:40.548712 20184 tablet_bootstrap.cc:492] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790: Bootstrap starting.
I20251213 08:58:40.550410 20185 tablet_bootstrap.cc:492] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad: Bootstrap starting.
I20251213 08:58:40.555868 20184 tablet_bootstrap.cc:654] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790: Neither blocks nor log segments found. Creating new log.
I20251213 08:58:40.555861 20186 tablet_bootstrap.cc:492] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79: Bootstrap starting.
I20251213 08:58:40.558024 20185 tablet_bootstrap.cc:654] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad: Neither blocks nor log segments found. Creating new log.
I20251213 08:58:40.562002 20184 tablet_bootstrap.cc:492] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790: No bootstrap required, opened a new log
I20251213 08:58:40.562448 20184 ts_tablet_manager.cc:1403] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790: Time spent bootstrapping tablet: real 0.014s user 0.011s sys 0.002s
I20251213 08:58:40.562644 20186 tablet_bootstrap.cc:654] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79: Neither blocks nor log segments found. Creating new log.
I20251213 08:58:40.564405 20185 tablet_bootstrap.cc:492] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad: No bootstrap required, opened a new log
I20251213 08:58:40.564955 20185 ts_tablet_manager.cc:1403] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.009s
I20251213 08:58:40.565529 20184 raft_consensus.cc:359] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:40.566322 20184 raft_consensus.cc:385] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:58:40.566758 20184 raft_consensus.cc:740] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95695f48d06342a2b8f760e3001d3790, State: Initialized, Role: FOLLOWER
I20251213 08:58:40.568012 20186 tablet_bootstrap.cc:492] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79: No bootstrap required, opened a new log
I20251213 08:58:40.568396 20186 ts_tablet_manager.cc:1403] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79: Time spent bootstrapping tablet: real 0.013s user 0.009s sys 0.001s
I20251213 08:58:40.567844 20184 consensus_queue.cc:260] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:40.568482 20185 raft_consensus.cc:359] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:40.569526 20185 raft_consensus.cc:385] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:58:40.569981 20185 raft_consensus.cc:740] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3421c084c4947d8a570edc0c7d913ad, State: Initialized, Role: FOLLOWER
I20251213 08:58:40.570775 20014 heartbeater.cc:499] Master 127.17.169.190:35231 was elected leader, sending a full tablet report...
I20251213 08:58:40.570919 20185 consensus_queue.cc:260] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:40.571355 20184 ts_tablet_manager.cc:1434] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790: Time spent starting tablet: real 0.009s user 0.006s sys 0.002s
I20251213 08:58:40.571375 20186 raft_consensus.cc:359] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:40.572261 20186 raft_consensus.cc:385] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:58:40.572574 20186 raft_consensus.cc:740] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0876bd2980b4442fb646de497ee70b79, State: Initialized, Role: FOLLOWER
I20251213 08:58:40.573647 20163 heartbeater.cc:499] Master 127.17.169.190:35231 was elected leader, sending a full tablet report...
I20251213 08:58:40.574373 20185 ts_tablet_manager.cc:1434] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad: Time spent starting tablet: real 0.009s user 0.005s sys 0.004s
I20251213 08:58:40.574074 20186 consensus_queue.cc:260] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:40.577620 20089 heartbeater.cc:499] Master 127.17.169.190:35231 was elected leader, sending a full tablet report...
I20251213 08:58:40.578430 20186 ts_tablet_manager.cc:1434] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79: Time spent starting tablet: real 0.010s user 0.003s sys 0.007s
I20251213 08:58:40.850538 20191 raft_consensus.cc:493] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251213 08:58:40.851012 20191 raft_consensus.cc:515] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:40.852806 20191 leader_election.cc:290] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 95695f48d06342a2b8f760e3001d3790 (127.17.169.129:43929), 0876bd2980b4442fb646de497ee70b79 (127.17.169.130:35039)
I20251213 08:58:40.861863 19989 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c622e37af7354d1f9b556f7ccb4c2ee1" candidate_uuid: "b3421c084c4947d8a570edc0c7d913ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95695f48d06342a2b8f760e3001d3790" is_pre_election: true
I20251213 08:58:40.862500 20064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c622e37af7354d1f9b556f7ccb4c2ee1" candidate_uuid: "b3421c084c4947d8a570edc0c7d913ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0876bd2980b4442fb646de497ee70b79" is_pre_election: true
I20251213 08:58:40.862676 19989 raft_consensus.cc:2468] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3421c084c4947d8a570edc0c7d913ad in term 0.
I20251213 08:58:40.863160 20064 raft_consensus.cc:2468] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3421c084c4947d8a570edc0c7d913ad in term 0.
I20251213 08:58:40.863924 20102 leader_election.cc:304] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95695f48d06342a2b8f760e3001d3790, b3421c084c4947d8a570edc0c7d913ad; no voters:
I20251213 08:58:40.864728 20191 raft_consensus.cc:2804] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251213 08:58:40.865087 20191 raft_consensus.cc:493] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251213 08:58:40.865347 20191 raft_consensus.cc:3060] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:58:40.870523 20191 raft_consensus.cc:515] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:40.871918 20191 leader_election.cc:290] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [CANDIDATE]: Term 1 election: Requested vote from peers 95695f48d06342a2b8f760e3001d3790 (127.17.169.129:43929), 0876bd2980b4442fb646de497ee70b79 (127.17.169.130:35039)
I20251213 08:58:40.872723 19989 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c622e37af7354d1f9b556f7ccb4c2ee1" candidate_uuid: "b3421c084c4947d8a570edc0c7d913ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "95695f48d06342a2b8f760e3001d3790"
I20251213 08:58:40.872929 20064 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c622e37af7354d1f9b556f7ccb4c2ee1" candidate_uuid: "b3421c084c4947d8a570edc0c7d913ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0876bd2980b4442fb646de497ee70b79"
I20251213 08:58:40.873178 19989 raft_consensus.cc:3060] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:58:40.873382 20064 raft_consensus.cc:3060] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:58:40.877998 20064 raft_consensus.cc:2468] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3421c084c4947d8a570edc0c7d913ad in term 1.
I20251213 08:58:40.878008 19989 raft_consensus.cc:2468] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3421c084c4947d8a570edc0c7d913ad in term 1.
I20251213 08:58:40.879217 20101 leader_election.cc:304] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0876bd2980b4442fb646de497ee70b79, b3421c084c4947d8a570edc0c7d913ad; no voters:
I20251213 08:58:40.879876 20191 raft_consensus.cc:2804] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:58:40.880671 20191 raft_consensus.cc:697] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 1 LEADER]: Becoming Leader. State: Replica: b3421c084c4947d8a570edc0c7d913ad, State: Running, Role: LEADER
I20251213 08:58:40.881424 20191 consensus_queue.cc:237] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:40.888684 19885 catalog_manager.cc:5654] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad reported cstate change: term changed from 0 to 1, leader changed from <none> to b3421c084c4947d8a570edc0c7d913ad (127.17.169.131). New cstate: current_term: 1 leader_uuid: "b3421c084c4947d8a570edc0c7d913ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } health_report { overall_health: UNKNOWN } } }
I20251213 08:58:40.902202 18086 test_util.cc:276] Using random seed: -1784810464
I20251213 08:58:40.925180 18086 test_workload.cc:405] TestWorkload: Skipping table creation because table test-workload already exists
I20251213 08:58:41.029421 20064 raft_consensus.cc:1275] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 1 FOLLOWER]: Refusing update from remote peer b3421c084c4947d8a570edc0c7d913ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:58:41.029423 19989 raft_consensus.cc:1275] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Refusing update from remote peer b3421c084c4947d8a570edc0c7d913ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:58:41.031317 20194 consensus_queue.cc:1048] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251213 08:58:41.032099 20191 consensus_queue.cc:1048] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251213 08:58:41.194012 19876 maintenance_manager.cc:419] P 149552c1ca614c8ab58059cb417e913b: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.003577
I20251213 08:58:41.429976 19875 maintenance_manager.cc:643] P 149552c1ca614c8ab58059cb417e913b: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.235s user 0.043s sys 0.012s Metrics: {"bytes_written":6635,"cfile_init":1,"dirs.queue_time_us":32780,"dirs.run_cpu_time_us":325,"dirs.run_wall_time_us":1285,"drs_written":1,"lbm_read_time_us":414,"lbm_reads_lt_1ms":4,"lbm_write_time_us":23531,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":5,"thread_start_us":11519,"threads_started":1}
I20251213 08:58:41.432951 19876 maintenance_manager.cc:419] P 149552c1ca614c8ab58059cb417e913b: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 488 bytes on disk
I20251213 08:58:41.434990 19875 maintenance_manager.cc:643] P 149552c1ca614c8ab58059cb417e913b: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":353,"lbm_reads_lt_1ms":4}
I20251213 08:58:41.580538 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.121360
I20251213 08:58:41.678035 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.121360
I20251213 08:58:41.728192 20015 maintenance_manager.cc:419] P 95695f48d06342a2b8f760e3001d3790: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.121360
I20251213 08:58:42.404269 19956 maintenance_manager.cc:643] P 95695f48d06342a2b8f760e3001d3790: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.668s user 0.331s sys 0.052s Metrics: {"bytes_written":35735,"cfile_init":1,"dirs.queue_time_us":6856,"dirs.run_cpu_time_us":301,"dirs.run_wall_time_us":11116,"drs_written":1,"lbm_read_time_us":289,"lbm_reads_lt_1ms":4,"lbm_write_time_us":39475,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":22,"peak_mem_usage":0,"rows_written":1200,"spinlock_wait_cycles":67328,"thread_start_us":7088,"threads_started":1}
I20251213 08:58:42.407344 20015 maintenance_manager.cc:419] P 95695f48d06342a2b8f760e3001d3790: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 10440 bytes on disk
I20251213 08:58:42.409047 19956 maintenance_manager.cc:643] P 95695f48d06342a2b8f760e3001d3790: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":298,"lbm_reads_lt_1ms":4}
I20251213 08:58:42.487555 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.901s user 0.291s sys 0.065s Metrics: {"bytes_written":26739,"cfile_init":1,"dirs.queue_time_us":61042,"dirs.run_cpu_time_us":306,"dirs.run_wall_time_us":2579,"drs_written":1,"lbm_read_time_us":267,"lbm_reads_lt_1ms":4,"lbm_write_time_us":6066,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":22,"peak_mem_usage":0,"rows_written":850,"spinlock_wait_cycles":335616,"thread_start_us":60742,"threads_started":1}
I20251213 08:58:42.490525 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 7449 bytes on disk
I20251213 08:58:42.492218 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":296,"lbm_reads_lt_1ms":4}
I20251213 08:58:42.515048 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.836s user 0.375s sys 0.050s Metrics: {"bytes_written":33151,"cfile_init":1,"dirs.queue_time_us":83546,"dirs.run_cpu_time_us":295,"dirs.run_wall_time_us":9559,"drs_written":1,"lbm_read_time_us":281,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2521,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1100,"thread_start_us":92085,"threads_started":1}
I20251213 08:58:42.527633 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 9605 bytes on disk
I20251213 08:58:42.529332 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":281,"lbm_reads_lt_1ms":4}
I20251213 08:58:43.428990 20015 maintenance_manager.cc:419] P 95695f48d06342a2b8f760e3001d3790: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.486106
I20251213 08:58:43.498591 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.486106
I20251213 08:58:43.535323 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.486106
I20251213 08:58:44.543149 19956 maintenance_manager.cc:643] P 95695f48d06342a2b8f760e3001d3790: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.113s user 0.725s sys 0.028s Metrics: {"bytes_written":78252,"cfile_init":1,"dirs.queue_time_us":56075,"dirs.run_cpu_time_us":339,"dirs.run_wall_time_us":1736,"drs_written":1,"lbm_read_time_us":313,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3434,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2700,"thread_start_us":62359,"threads_started":1}
I20251213 08:58:44.546718 20015 maintenance_manager.cc:419] P 95695f48d06342a2b8f760e3001d3790: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 24150 bytes on disk
I20251213 08:58:44.548507 19956 maintenance_manager.cc:643] P 95695f48d06342a2b8f760e3001d3790: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":294,"lbm_reads_lt_1ms":4}
I20251213 08:58:44.551100 20015 maintenance_manager.cc:419] P 95695f48d06342a2b8f760e3001d3790: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014177
W20251213 08:58:44.766409 20201 meta_cache.cc:1261] Time spent looking up entry by key: real 0.052s user 0.004s sys 0.000s
I20251213 08:58:45.261250 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.725s user 0.799s sys 0.004s Metrics: {"bytes_written":86004,"cfile_init":1,"dirs.queue_time_us":9169,"dirs.run_cpu_time_us":360,"dirs.run_wall_time_us":10943,"drs_written":1,"lbm_read_time_us":461,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3409,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3000,"spinlock_wait_cycles":247552,"thread_start_us":9373,"threads_started":1}
I20251213 08:58:45.265152 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014130
I20251213 08:58:45.302812 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.803s user 0.865s sys 0.010s Metrics: {"bytes_written":91207,"cfile_init":1,"dirs.queue_time_us":7972,"dirs.run_cpu_time_us":347,"dirs.run_wall_time_us":9888,"drs_written":1,"lbm_read_time_us":298,"lbm_reads_lt_1ms":4,"lbm_write_time_us":18387,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":3200,"thread_start_us":9221,"threads_started":1}
I20251213 08:58:45.306368 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014065
I20251213 08:58:45.540544 19956 maintenance_manager.cc:643] P 95695f48d06342a2b8f760e3001d3790: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.989s user 0.740s sys 0.012s Metrics: {"bytes_written":113176,"cfile_cache_hit":10,"cfile_cache_hit_bytes":71410,"cfile_cache_miss":28,"cfile_cache_miss_bytes":199460,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":6853,"dirs.run_cpu_time_us":409,"dirs.run_wall_time_us":22227,"drs_written":1,"lbm_read_time_us":4177,"lbm_reads_lt_1ms":48,"lbm_write_time_us":3604,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":62076,"rows_written":3900,"thread_start_us":7106,"threads_started":1}
I20251213 08:58:45.552701 20015 maintenance_manager.cc:419] P 95695f48d06342a2b8f760e3001d3790: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.486106
I20251213 08:58:46.631103 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.313s user 0.814s sys 0.063s Metrics: {"bytes_written":116809,"cfile_cache_hit":10,"cfile_cache_hit_bytes":74150,"cfile_cache_miss":28,"cfile_cache_miss_bytes":207716,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":9817,"dirs.run_cpu_time_us":1804,"dirs.run_wall_time_us":18650,"drs_written":1,"lbm_read_time_us":4885,"lbm_reads_lt_1ms":52,"lbm_write_time_us":3708,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":63066,"rows_written":4050,"spinlock_wait_cycles":2432,"thread_start_us":69265,"threads_started":3}
I20251213 08:58:46.635581 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 36474 bytes on disk
I20251213 08:58:46.661381 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":284,"lbm_reads_lt_1ms":4}
I20251213 08:58:46.664961 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.486106
I20251213 08:58:46.826067 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.560s user 0.860s sys 0.051s Metrics: {"bytes_written":118419,"cfile_cache_hit":10,"cfile_cache_hit_bytes":75060,"cfile_cache_miss":28,"cfile_cache_miss_bytes":209850,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":88259,"dirs.run_cpu_time_us":364,"dirs.run_wall_time_us":6353,"drs_written":1,"lbm_read_time_us":4507,"lbm_reads_lt_1ms":52,"lbm_write_time_us":4040,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":65226,"rows_written":4100,"spinlock_wait_cycles":1920,"thread_start_us":87715,"threads_started":1}
I20251213 08:58:46.829784 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 36911 bytes on disk
I20251213 08:58:46.831844 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":282,"lbm_reads_lt_1ms":4}
I20251213 08:58:46.833906 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.486106
I20251213 08:58:47.119326 19956 maintenance_manager.cc:643] P 95695f48d06342a2b8f760e3001d3790: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.566s user 0.663s sys 0.048s Metrics: {"bytes_written":70513,"cfile_init":1,"dirs.queue_time_us":15200,"dirs.run_cpu_time_us":337,"dirs.run_wall_time_us":6966,"drs_written":1,"lbm_read_time_us":312,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3490,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":2400,"spinlock_wait_cycles":91330688,"thread_start_us":14791,"threads_started":1}
I20251213 08:58:47.123548 20015 maintenance_manager.cc:419] P 95695f48d06342a2b8f760e3001d3790: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014774
W20251213 08:58:47.816324 20203 meta_cache.cc:1261] Time spent looking up entry by key: real 0.062s user 0.004s sys 0.000s
I20251213 08:58:47.997632 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.332s user 0.875s sys 0.076s Metrics: {"bytes_written":94784,"cfile_init":1,"dirs.queue_time_us":71444,"dirs.run_cpu_time_us":312,"dirs.run_wall_time_us":4436,"drs_written":1,"lbm_read_time_us":307,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3379,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3350,"spinlock_wait_cycles":9088,"thread_start_us":71134,"threads_started":1}
I20251213 08:58:48.001255 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014728
I20251213 08:58:48.443986 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.609s user 0.907s sys 0.015s Metrics: {"bytes_written":101781,"cfile_init":1,"dirs.queue_time_us":10161,"dirs.run_cpu_time_us":291,"dirs.run_wall_time_us":4999,"drs_written":1,"lbm_read_time_us":264,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3567,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":3450,"thread_start_us":9888,"threads_started":1}
I20251213 08:58:48.448089 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 30927 bytes on disk
I20251213 08:58:48.455482 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":298,"lbm_reads_lt_1ms":4}
I20251213 08:58:48.458434 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014723
W20251213 08:58:48.487857 20204 meta_cache.cc:1261] Time spent looking up entry by key: real 0.054s user 0.000s sys 0.000s
I20251213 08:58:49.509115 19956 maintenance_manager.cc:643] P 95695f48d06342a2b8f760e3001d3790: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 2.385s user 1.248s sys 0.060s Metrics: {"bytes_written":180746,"cfile_cache_hit":10,"cfile_cache_hit_bytes":115233,"cfile_cache_miss":30,"cfile_cache_miss_bytes":322149,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":70894,"dirs.run_cpu_time_us":1510,"dirs.run_wall_time_us":7316,"drs_written":1,"lbm_read_time_us":15203,"lbm_reads_1-10_ms":3,"lbm_reads_lt_1ms":55,"lbm_write_time_us":4544,"lbm_writes_lt_1ms":28,"num_input_rowsets":2,"peak_mem_usage":97581,"rows_written":6300,"thread_start_us":78132,"threads_started":4}
I20251213 08:58:49.522149 20015 maintenance_manager.cc:419] P 95695f48d06342a2b8f760e3001d3790: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 57714 bytes on disk
I20251213 08:58:49.523942 19956 maintenance_manager.cc:643] P 95695f48d06342a2b8f760e3001d3790: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":300,"lbm_reads_lt_1ms":4}
I20251213 08:58:49.546370 20015 maintenance_manager.cc:419] P 95695f48d06342a2b8f760e3001d3790: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.486106
I20251213 08:58:49.740513 18086 stop_tablet-itest.cc:98] Stopping T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790
W20251213 08:58:49.754387 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 0 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:49.755287 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:49.757246 20102 consensus_peers.cc:597] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad -> Peer 95695f48d06342a2b8f760e3001d3790 (127.17.169.129:43929): Couldn't send request to peer 95695f48d06342a2b8f760e3001d3790. Status: Illegal state: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped. This is attempt 1: this message will repeat every 5th retry.
I20251213 08:58:49.791260 18086 stop_tablet-itest.cc:281] 10400 / 20300 rows inserted
I20251213 08:58:50.070478 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 2.068s user 1.435s sys 0.035s Metrics: {"bytes_written":214073,"cfile_cache_hit":10,"cfile_cache_hit_bytes":135313,"cfile_cache_miss":30,"cfile_cache_miss_bytes":378101,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":3854,"dirs.run_cpu_time_us":276,"dirs.run_wall_time_us":11193,"drs_written":1,"lbm_read_time_us":6128,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":53,"lbm_write_time_us":5031,"lbm_writes_lt_1ms":30,"num_input_rowsets":2,"peak_mem_usage":111061,"rows_written":7400,"thread_start_us":3541,"threads_started":1}
I20251213 08:58:50.084079 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 68376 bytes on disk
I20251213 08:58:50.085842 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":317,"lbm_reads_lt_1ms":4}
I20251213 08:58:50.088222 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.486106
W20251213 08:58:50.221511 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 11 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:50.260052 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:50.682108 19956 tablet_replica_mm_ops.cc:214] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790: failed to flush MRS: Aborted: MVCC is closed
I20251213 08:58:50.682988 19956 maintenance_manager.cc:643] P 95695f48d06342a2b8f760e3001d3790: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.133s user 0.750s sys 0.015s Metrics: {"cfile_init":1,"dirs.queue_time_us":9486,"dirs.run_cpu_time_us":311,"dirs.run_wall_time_us":7288,"lbm_read_time_us":256,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3897,"lbm_writes_lt_1ms":26,"thread_start_us":9688,"threads_started":1}
I20251213 08:58:50.760438 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 2.301s user 1.444s sys 0.028s Metrics: {"bytes_written":218056,"cfile_cache_hit":11,"cfile_cache_hit_bytes":151496,"cfile_cache_miss":29,"cfile_cache_miss_bytes":373588,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":6859,"dirs.run_cpu_time_us":332,"dirs.run_wall_time_us":10514,"drs_written":1,"lbm_read_time_us":6016,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":48,"lbm_write_time_us":5035,"lbm_writes_lt_1ms":30,"num_input_rowsets":2,"peak_mem_usage":114996,"rows_written":7550,"spinlock_wait_cycles":33536,"thread_start_us":7060,"threads_started":1}
I20251213 08:58:50.763971 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.486106
W20251213 08:58:50.782051 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 25 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:50.783807 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:50.795966 18086 stop_tablet-itest.cc:281] 11550 / 20300 rows inserted
I20251213 08:58:51.218677 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.129s user 0.897s sys 0.011s Metrics: {"bytes_written":94844,"cfile_init":1,"dirs.queue_time_us":15272,"dirs.run_cpu_time_us":313,"dirs.run_wall_time_us":4669,"drs_written":1,"lbm_read_time_us":278,"lbm_reads_lt_1ms":4,"lbm_write_time_us":5430,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":3350,"thread_start_us":14973,"threads_started":1}
I20251213 08:58:51.222429 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 29989 bytes on disk
I20251213 08:58:51.224326 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":288,"lbm_reads_lt_1ms":4}
I20251213 08:58:51.228271 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014646
W20251213 08:58:51.292107 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 37 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:51.294565 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:51.718741 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 48 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:51.721292 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:51.800393 18086 stop_tablet-itest.cc:281] 12900 / 20300 rows inserted
I20251213 08:58:52.096321 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 1.331s user 1.076s sys 0.024s Metrics: {"bytes_written":118415,"cfile_init":1,"dirs.queue_time_us":8070,"dirs.run_cpu_time_us":359,"dirs.run_wall_time_us":7472,"drs_written":1,"lbm_read_time_us":288,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3653,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":4100,"thread_start_us":8285,"threads_started":1}
I20251213 08:58:52.111665 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014725
W20251213 08:58:52.255861 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 64 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:52.272465 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:52.275381 20102 consensus_peers.cc:597] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad -> Peer 95695f48d06342a2b8f760e3001d3790 (127.17.169.129:43929): Couldn't send request to peer 95695f48d06342a2b8f760e3001d3790. Status: Illegal state: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped. This is attempt 6: this message will repeat every 5th retry.
W20251213 08:58:52.707406 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 78 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:52.725790 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:52.810590 18086 stop_tablet-itest.cc:281] 14500 / 20300 rows inserted
W20251213 08:58:53.359931 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 96 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:53.367646 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:53.773496 20203 meta_cache.cc:1261] Time spent looking up entry by key: real 0.059s user 0.004s sys 0.000s
I20251213 08:58:53.825659 18086 stop_tablet-itest.cc:281] 16150 / 20300 rows inserted
W20251213 08:58:53.922861 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 116 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:53.985522 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:54.335634 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 3.106s user 2.077s sys 0.073s Metrics: {"bytes_written":310928,"cfile_cache_hit":11,"cfile_cache_hit_bytes":223234,"cfile_cache_miss":33,"cfile_cache_miss_bytes":532096,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":11134,"dirs.run_cpu_time_us":1601,"dirs.run_wall_time_us":26899,"drs_written":1,"lbm_read_time_us":7475,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":52,"lbm_write_time_us":6039,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":32,"mutex_wait_us":5,"num_input_rowsets":2,"peak_mem_usage":163787,"rows_written":10750,"spinlock_wait_cycles":21888,"thread_start_us":193592,"threads_started":3}
I20251213 08:58:54.341203 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 100459 bytes on disk
I20251213 08:58:54.343222 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":291,"lbm_reads_lt_1ms":4}
I20251213 08:58:54.345311 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.972434
W20251213 08:58:54.642655 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 136 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:54.676316 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:54.800843 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 2.688s user 2.214s sys 0.068s Metrics: {"bytes_written":334384,"cfile_cache_hit":10,"cfile_cache_hit_bytes":216380,"cfile_cache_miss":34,"cfile_cache_miss_bytes":602258,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":11396,"dirs.run_cpu_time_us":318,"dirs.run_wall_time_us":6807,"drs_written":1,"lbm_read_time_us":9668,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":60,"lbm_write_time_us":6061,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":33,"num_input_rowsets":2,"peak_mem_usage":177512,"rows_written":11650,"thread_start_us":11627,"threads_started":1}
I20251213 08:58:54.804829 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=0.972434
I20251213 08:58:54.831012 18086 stop_tablet-itest.cc:281] 17750 / 20300 rows inserted
W20251213 08:58:55.032827 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 151 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:55.039788 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:55.042500 20102 consensus_peers.cc:597] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad -> Peer 95695f48d06342a2b8f760e3001d3790 (127.17.169.129:43929): Couldn't send request to peer 95695f48d06342a2b8f760e3001d3790. Status: Illegal state: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped. This is attempt 11: this message will repeat every 5th retry.
W20251213 08:58:55.702903 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 167 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:55.709823 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:55.834609 18086 stop_tablet-itest.cc:281] 19150 / 20300 rows inserted
W20251213 08:58:56.076797 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 180 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:56.090019 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
W20251213 08:58:56.626603 19988 raft_consensus.cc:1579] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Could not prepare op 'term: 1 index: 209' and following 193 ops. Status for this op: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:56.635689 19988 raft_consensus.cc:1595] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Rejecting Update request from peer b3421c084c4947d8a570edc0c7d913ad for term 1. Could not prepare a single op due to: Illegal state: Not initializing new op; the tablet is stopped
I20251213 08:58:56.813093 18086 tablet_server.cc:178] TabletServer@127.17.169.129:0 shutting down...
I20251213 08:58:56.863263 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:58:56.864056 18086 tablet_replica.cc:333] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790: stopping tablet replica
I20251213 08:58:56.864760 18086 raft_consensus.cc:2243] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:58:56.865485 18086 raft_consensus.cc:2272] T c622e37af7354d1f9b556f7ccb4c2ee1 P 95695f48d06342a2b8f760e3001d3790 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:56.865128 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 2.519s user 1.660s sys 0.036s Metrics: {"bytes_written":178117,"cfile_init":1,"dirs.queue_time_us":17952,"dirs.run_cpu_time_us":306,"dirs.run_wall_time_us":6712,"drs_written":1,"lbm_read_time_us":274,"lbm_reads_lt_1ms":4,"lbm_write_time_us":24468,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":6200,"thread_start_us":24789,"threads_started":1}
I20251213 08:58:56.869423 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1): 56816 bytes on disk
I20251213 08:58:56.871529 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: UndoDeltaBlockGCOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":424,"lbm_reads_lt_1ms":4}
I20251213 08:58:56.874389 20090 maintenance_manager.cc:419] P 0876bd2980b4442fb646de497ee70b79: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014629
I20251213 08:58:56.905320 18086 tablet_server.cc:195] TabletServer@127.17.169.129:0 shutdown complete.
W20251213 08:58:56.918653 20102 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.17.169.129:43929: connect: Connection refused (error 111) [suppressed 1 similar messages]
I20251213 08:58:56.924540 18086 tablet_server.cc:178] TabletServer@127.17.169.130:0 shutting down...
I20251213 08:58:57.032708 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: FlushMRSOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 2.211s user 1.690s sys 0.056s Metrics: {"bytes_written":178186,"cfile_init":1,"dirs.queue_time_us":34087,"dirs.run_cpu_time_us":306,"dirs.run_wall_time_us":2077,"drs_written":1,"lbm_read_time_us":271,"lbm_reads_lt_1ms":4,"lbm_write_time_us":4746,"lbm_writes_lt_1ms":28,"peak_mem_usage":0,"rows_written":6200,"spinlock_wait_cycles":26739456,"thread_start_us":33691,"threads_started":1}
I20251213 08:58:57.036235 20164 maintenance_manager.cc:419] P b3421c084c4947d8a570edc0c7d913ad: Scheduling CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1): perf score=1.014511
W20251213 08:58:57.245645 20101 consensus_peers.cc:597] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad -> Peer 0876bd2980b4442fb646de497ee70b79 (127.17.169.130:35039): Couldn't send request to peer 0876bd2980b4442fb646de497ee70b79. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20251213 08:58:57.340613 20102 consensus_peers.cc:597] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad -> Peer 95695f48d06342a2b8f760e3001d3790 (127.17.169.129:43929): Couldn't send request to peer 95695f48d06342a2b8f760e3001d3790. Status: Network error: Client connection negotiation failed: client connection to 127.17.169.129:43929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20251213 08:58:58.294883 20327 raft_consensus.cc:493] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b3421c084c4947d8a570edc0c7d913ad)
I20251213 08:58:58.295543 20327 raft_consensus.cc:515] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3421c084c4947d8a570edc0c7d913ad" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43485 } } peers { permanent_uuid: "95695f48d06342a2b8f760e3001d3790" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 43929 } } peers { permanent_uuid: "0876bd2980b4442fb646de497ee70b79" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 35039 } }
I20251213 08:58:58.304499 20327 leader_election.cc:290] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b3421c084c4947d8a570edc0c7d913ad (127.17.169.131:43485), 95695f48d06342a2b8f760e3001d3790 (127.17.169.129:43929)
I20251213 08:58:58.336457 20138 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "c622e37af7354d1f9b556f7ccb4c2ee1" candidate_uuid: "0876bd2980b4442fb646de497ee70b79" candidate_term: 2 candidate_status { last_received { term: 1 index: 412 } } ignore_live_leader: false dest_uuid: "b3421c084c4947d8a570edc0c7d913ad" is_pre_election: true
W20251213 08:58:58.339413 20028 leader_election.cc:336] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 95695f48d06342a2b8f760e3001d3790 (127.17.169.129:43929): Network error: Client connection negotiation failed: client connection to 127.17.169.129:43929: connect: Connection refused (error 111)
I20251213 08:58:58.339814 20028 leader_election.cc:304] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0876bd2980b4442fb646de497ee70b79; no voters: 95695f48d06342a2b8f760e3001d3790, b3421c084c4947d8a570edc0c7d913ad
I20251213 08:58:58.340512 20327 raft_consensus.cc:2749] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251213 08:58:58.937532 20031 maintenance_manager.cc:643] P 0876bd2980b4442fb646de497ee70b79: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 2.059s user 1.991s sys 0.052s Metrics: {"bytes_written":483070,"cfile_cache_hit":12,"cfile_cache_hit_bytes":385127,"cfile_cache_miss":36,"cfile_cache_miss_bytes":815325,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":4553,"dirs.run_cpu_time_us":1199,"dirs.run_wall_time_us":5483,"drs_written":1,"lbm_read_time_us":11170,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":54,"lbm_write_time_us":6663,"lbm_writes_lt_1ms":40,"mutex_wait_us":274,"num_input_rowsets":2,"peak_mem_usage":258328,"rows_written":16950,"spinlock_wait_cycles":39168,"thread_start_us":3894,"threads_started":6,"wal-append.queue_time_us":960}
I20251213 08:58:58.943006 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:58:58.944001 18086 tablet_replica.cc:333] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79: stopping tablet replica
I20251213 08:58:58.944808 18086 raft_consensus.cc:2243] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:58:58.963888 18086 raft_consensus.cc:2272] T c622e37af7354d1f9b556f7ccb4c2ee1 P 0876bd2980b4442fb646de497ee70b79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:59.000712 18086 tablet_server.cc:195] TabletServer@127.17.169.130:0 shutdown complete.
I20251213 08:58:59.015419 18086 tablet_server.cc:178] TabletServer@127.17.169.131:0 shutting down...
I20251213 08:58:59.174757 20105 maintenance_manager.cc:643] P b3421c084c4947d8a570edc0c7d913ad: CompactRowSetsOp(c622e37af7354d1f9b556f7ccb4c2ee1) complete. Timing: real 2.138s user 2.087s sys 0.048s Metrics: {"bytes_written":509703,"cfile_cache_hit":10,"cfile_cache_hit_bytes":335800,"cfile_cache_miss":40,"cfile_cache_miss_bytes":930442,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":4009,"dirs.run_cpu_time_us":1288,"dirs.run_wall_time_us":5529,"drs_written":1,"lbm_read_time_us":10136,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":66,"lbm_write_time_us":7068,"lbm_writes_lt_1ms":42,"mutex_wait_us":90,"num_input_rowsets":2,"peak_mem_usage":272051,"rows_written":17850,"thread_start_us":3640,"threads_started":6,"wal-append.queue_time_us":827}
I20251213 08:58:59.176771 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:58:59.177632 18086 tablet_replica.cc:333] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad: stopping tablet replica
I20251213 08:58:59.178139 18086 raft_consensus.cc:2243] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:58:59.186950 18086 raft_consensus.cc:2272] T c622e37af7354d1f9b556f7ccb4c2ee1 P b3421c084c4947d8a570edc0c7d913ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:59.213394 18086 tablet_server.cc:195] TabletServer@127.17.169.131:0 shutdown complete.
I20251213 08:58:59.228865 18086 master.cc:561] Master@127.17.169.190:35231 shutting down...
I20251213 08:58:59.253214 18086 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:58:59.253924 18086 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:58:59.254309 18086 tablet_replica.cc:333] T 00000000000000000000000000000000 P 149552c1ca614c8ab58059cb417e913b: stopping tablet replica
I20251213 08:58:59.276145 18086 master.cc:583] Master@127.17.169.190:35231 shutdown complete.
[ OK ] StopTablets/StopTabletITest.TestStoppedTabletsDontWrite/1 (19444 ms)
[ RUN ] StopTablets/StopTabletITest.TestShutdownWhileWriting/0
I20251213 08:58:59.316859 18086 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.17.169.190:45943
I20251213 08:58:59.318679 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:59.324725 20346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:59.324954 20347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:59.326107 20349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:59.326725 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:59.327587 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:59.327781 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:59.327889 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616339327879 us; error 0 us; skew 500 ppm
I20251213 08:58:59.328363 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:59.331583 18086 webserver.cc:492] Webserver started at http://127.17.169.190:44063/ using document root <none> and password file <none>
I20251213 08:58:59.332340 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:59.332589 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:59.332826 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:59.333940 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/master-0-root/instance:
uuid: "815a02557c494c77a1e3fa3112b49991"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:58:59.339130 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20251213 08:58:59.343891 20354 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.344938 18086 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20251213 08:58:59.345335 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/master-0-root
uuid: "815a02557c494c77a1e3fa3112b49991"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:58:59.345633 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:59.369117 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:59.370445 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:59.420105 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.190:45943
I20251213 08:58:59.420203 20405 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.190:45943 every 8 connection(s)
I20251213 08:58:59.425071 20406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:58:59.439005 20406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991: Bootstrap starting.
I20251213 08:58:59.445240 20406 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991: Neither blocks nor log segments found. Creating new log.
I20251213 08:58:59.450572 20406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991: No bootstrap required, opened a new log
I20251213 08:58:59.453172 20406 raft_consensus.cc:359] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815a02557c494c77a1e3fa3112b49991" member_type: VOTER }
I20251213 08:58:59.453652 20406 raft_consensus.cc:385] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:58:59.453922 20406 raft_consensus.cc:740] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815a02557c494c77a1e3fa3112b49991, State: Initialized, Role: FOLLOWER
I20251213 08:58:59.454807 20406 consensus_queue.cc:260] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815a02557c494c77a1e3fa3112b49991" member_type: VOTER }
I20251213 08:58:59.455327 20406 raft_consensus.cc:399] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:58:59.455705 20406 raft_consensus.cc:493] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:58:59.456012 20406 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:58:59.461843 20406 raft_consensus.cc:515] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815a02557c494c77a1e3fa3112b49991" member_type: VOTER }
I20251213 08:58:59.462529 20406 leader_election.cc:304] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 815a02557c494c77a1e3fa3112b49991; no voters:
I20251213 08:58:59.463922 20406 leader_election.cc:290] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:58:59.464402 20409 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:58:59.466051 20409 raft_consensus.cc:697] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 1 LEADER]: Becoming Leader. State: Replica: 815a02557c494c77a1e3fa3112b49991, State: Running, Role: LEADER
I20251213 08:58:59.466967 20409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815a02557c494c77a1e3fa3112b49991" member_type: VOTER }
I20251213 08:58:59.467398 20406 sys_catalog.cc:565] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [sys.catalog]: configured and running, proceeding with master startup.
I20251213 08:58:59.472374 20410 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "815a02557c494c77a1e3fa3112b49991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815a02557c494c77a1e3fa3112b49991" member_type: VOTER } }
I20251213 08:58:59.473476 20410 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [sys.catalog]: This master's current role is: LEADER
I20251213 08:58:59.473511 20411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 815a02557c494c77a1e3fa3112b49991. Latest consensus state: current_term: 1 leader_uuid: "815a02557c494c77a1e3fa3112b49991" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815a02557c494c77a1e3fa3112b49991" member_type: VOTER } }
I20251213 08:58:59.474597 20411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [sys.catalog]: This master's current role is: LEADER
I20251213 08:58:59.478439 20417 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251213 08:58:59.487670 20417 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251213 08:58:59.488837 18086 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251213 08:58:59.497932 20417 catalog_manager.cc:1357] Generated new cluster ID: f95aeb96c7ec4abaa071490505cb7745
I20251213 08:58:59.498370 20417 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251213 08:58:59.519862 20417 catalog_manager.cc:1380] Generated new certificate authority record
I20251213 08:58:59.521416 20417 catalog_manager.cc:1514] Loading token signing keys...
I20251213 08:58:59.535220 20417 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991: Generated new TSK 0
I20251213 08:58:59.535971 20417 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251213 08:58:59.555936 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:59.563001 20427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:59.564404 20428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:59.567371 18086 server_base.cc:1047] running on GCE node
W20251213 08:58:59.567502 20430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:59.568701 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:59.568954 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:59.569118 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616339569098 us; error 0 us; skew 500 ppm
I20251213 08:58:59.569689 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:59.572418 18086 webserver.cc:492] Webserver started at http://127.17.169.129:46791/ using document root <none> and password file <none>
I20251213 08:58:59.573029 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:59.573235 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:59.573503 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:59.574826 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-0-root/instance:
uuid: "0b945a9ff9c4400b80f54337d0586e91"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:58:59.579815 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20251213 08:58:59.583882 20435 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.584712 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20251213 08:58:59.585011 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-0-root
uuid: "0b945a9ff9c4400b80f54337d0586e91"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:58:59.585311 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:59.609583 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:59.611070 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:59.612766 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:58:59.615432 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:58:59.615653 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.615906 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:58:59.616081 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.672185 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.129:40351
I20251213 08:58:59.672312 20497 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.129:40351 every 8 connection(s)
I20251213 08:58:59.678121 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:59.686815 20502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:59.688052 20503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:59.690652 20505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:59.692602 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:59.693594 20498 heartbeater.cc:344] Connected to a master server at 127.17.169.190:45943
I20251213 08:58:59.693779 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20251213 08:58:59.693971 20498 heartbeater.cc:461] Registering TS with master...
W20251213 08:58:59.694032 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:59.694438 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616339694368 us; error 0 us; skew 500 ppm
I20251213 08:58:59.694943 20498 heartbeater.cc:507] Master 127.17.169.190:45943 requested a full tablet report, sending...
I20251213 08:58:59.695103 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:59.697458 20371 ts_manager.cc:194] Registered new tserver with Master: 0b945a9ff9c4400b80f54337d0586e91 (127.17.169.129:40351)
I20251213 08:58:59.698385 18086 webserver.cc:492] Webserver started at http://127.17.169.130:37175/ using document root <none> and password file <none>
I20251213 08:58:59.699129 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:59.699398 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:59.699743 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:59.700259 20371 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33238
I20251213 08:58:59.701406 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-1-root/instance:
uuid: "3a18718f5cdd421e8960146e869ad460"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:58:59.706872 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20251213 08:58:59.710724 20510 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.711618 18086 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20251213 08:58:59.711923 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-1-root
uuid: "3a18718f5cdd421e8960146e869ad460"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:58:59.712201 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:59.723879 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:59.725235 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:59.726955 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:58:59.729482 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:58:59.729677 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.729943 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:58:59.730089 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.778681 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.130:41379
I20251213 08:58:59.778831 20572 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.130:41379 every 8 connection(s)
I20251213 08:58:59.784361 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:58:59.794576 20576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:59.795413 20577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:58:59.797663 20579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:58:59.798569 18086 server_base.cc:1047] running on GCE node
I20251213 08:58:59.799818 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:58:59.800102 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:58:59.800137 20573 heartbeater.cc:344] Connected to a master server at 127.17.169.190:45943
I20251213 08:58:59.800328 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616339800309 us; error 0 us; skew 500 ppm
I20251213 08:58:59.800751 20573 heartbeater.cc:461] Registering TS with master...
I20251213 08:58:59.801138 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:58:59.801644 20573 heartbeater.cc:507] Master 127.17.169.190:45943 requested a full tablet report, sending...
I20251213 08:58:59.804394 18086 webserver.cc:492] Webserver started at http://127.17.169.131:42649/ using document root <none> and password file <none>
I20251213 08:58:59.804394 20371 ts_manager.cc:194] Registered new tserver with Master: 3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379)
I20251213 08:58:59.805281 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:58:59.805588 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:58:59.805938 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:58:59.806668 20371 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33254
I20251213 08:58:59.808847 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-2-root/instance:
uuid: "8e8c8b42dfa94db2860263cebcad3f7c"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:58:59.814240 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20251213 08:58:59.818219 20584 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.819065 18086 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20251213 08:58:59.819370 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-2-root
uuid: "8e8c8b42dfa94db2860263cebcad3f7c"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:58:59.819769 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:58:59.836867 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:58:59.838284 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:58:59.840067 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:58:59.842716 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:58:59.842953 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.843219 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:58:59.843376 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:58:59.893385 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.131:37829
I20251213 08:58:59.893554 20646 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.131:37829 every 8 connection(s)
I20251213 08:58:59.908224 20647 heartbeater.cc:344] Connected to a master server at 127.17.169.190:45943
I20251213 08:58:59.908713 20647 heartbeater.cc:461] Registering TS with master...
I20251213 08:58:59.909492 20647 heartbeater.cc:507] Master 127.17.169.190:45943 requested a full tablet report, sending...
I20251213 08:58:59.911566 20371 ts_manager.cc:194] Registered new tserver with Master: 8e8c8b42dfa94db2860263cebcad3f7c (127.17.169.131:37829)
I20251213 08:58:59.911847 18086 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.01498665s
I20251213 08:58:59.913161 20371 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33266
I20251213 08:58:59.951269 18086 test_util.cc:276] Using random seed: -1765761401
I20251213 08:58:59.976608 20371 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33286:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251213 08:58:59.978961 20371 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251213 08:59:00.039530 20538 tablet_service.cc:1505] Processing CreateTablet for tablet a56e271ad82a4526a70677cf90fb9705 (DEFAULT_TABLE table=test-workload [id=cf336fec59d447d2967dc23d9507150f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:59:00.041771 20538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56e271ad82a4526a70677cf90fb9705. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:59:00.043465 20463 tablet_service.cc:1505] Processing CreateTablet for tablet a56e271ad82a4526a70677cf90fb9705 (DEFAULT_TABLE table=test-workload [id=cf336fec59d447d2967dc23d9507150f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:59:00.044945 20463 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56e271ad82a4526a70677cf90fb9705. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:59:00.050644 20612 tablet_service.cc:1505] Processing CreateTablet for tablet a56e271ad82a4526a70677cf90fb9705 (DEFAULT_TABLE table=test-workload [id=cf336fec59d447d2967dc23d9507150f]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:59:00.052383 20612 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a56e271ad82a4526a70677cf90fb9705. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:59:00.065886 20668 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: Bootstrap starting.
I20251213 08:59:00.072345 20669 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: Bootstrap starting.
I20251213 08:59:00.074036 20670 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Bootstrap starting.
I20251213 08:59:00.076494 20668 tablet_bootstrap.cc:654] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: Neither blocks nor log segments found. Creating new log.
I20251213 08:59:00.081693 20670 tablet_bootstrap.cc:654] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Neither blocks nor log segments found. Creating new log.
I20251213 08:59:00.083868 20669 tablet_bootstrap.cc:654] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: Neither blocks nor log segments found. Creating new log.
I20251213 08:59:00.088200 20668 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: No bootstrap required, opened a new log
I20251213 08:59:00.088719 20668 ts_tablet_manager.cc:1403] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.007s
I20251213 08:59:00.089218 20670 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: No bootstrap required, opened a new log
I20251213 08:59:00.089773 20670 ts_tablet_manager.cc:1403] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Time spent bootstrapping tablet: real 0.016s user 0.010s sys 0.003s
I20251213 08:59:00.090615 20669 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: No bootstrap required, opened a new log
I20251213 08:59:00.091188 20669 ts_tablet_manager.cc:1403] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.005s
I20251213 08:59:00.091322 20668 raft_consensus.cc:359] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:00.092021 20668 raft_consensus.cc:385] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:59:00.092281 20668 raft_consensus.cc:740] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a18718f5cdd421e8960146e869ad460, State: Initialized, Role: FOLLOWER
I20251213 08:59:00.092973 20668 consensus_queue.cc:260] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:00.093168 20670 raft_consensus.cc:359] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:00.093987 20670 raft_consensus.cc:385] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:59:00.094369 20670 raft_consensus.cc:740] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e8c8b42dfa94db2860263cebcad3f7c, State: Initialized, Role: FOLLOWER
I20251213 08:59:00.094311 20669 raft_consensus.cc:359] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:00.095199 20669 raft_consensus.cc:385] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:59:00.095536 20669 raft_consensus.cc:740] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b945a9ff9c4400b80f54337d0586e91, State: Initialized, Role: FOLLOWER
I20251213 08:59:00.095922 20573 heartbeater.cc:499] Master 127.17.169.190:45943 was elected leader, sending a full tablet report...
I20251213 08:59:00.095475 20670 consensus_queue.cc:260] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:00.096491 20668 ts_tablet_manager.cc:1434] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: Time spent starting tablet: real 0.007s user 0.006s sys 0.003s
I20251213 08:59:00.097358 20669 consensus_queue.cc:260] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:00.102875 20647 heartbeater.cc:499] Master 127.17.169.190:45943 was elected leader, sending a full tablet report...
I20251213 08:59:00.103802 20498 heartbeater.cc:499] Master 127.17.169.190:45943 was elected leader, sending a full tablet report...
I20251213 08:59:00.105327 20669 ts_tablet_manager.cc:1434] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20251213 08:59:00.106360 20670 ts_tablet_manager.cc:1434] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20251213 08:59:00.115830 20676 raft_consensus.cc:493] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251213 08:59:00.116461 20676 raft_consensus.cc:515] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:00.119355 20676 leader_election.cc:290] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379), 8e8c8b42dfa94db2860263cebcad3f7c (127.17.169.131:37829)
I20251213 08:59:00.129592 20548 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56e271ad82a4526a70677cf90fb9705" candidate_uuid: "0b945a9ff9c4400b80f54337d0586e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a18718f5cdd421e8960146e869ad460" is_pre_election: true
I20251213 08:59:00.130496 20548 raft_consensus.cc:2468] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b945a9ff9c4400b80f54337d0586e91 in term 0.
I20251213 08:59:00.131661 20439 leader_election.cc:304] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b945a9ff9c4400b80f54337d0586e91, 3a18718f5cdd421e8960146e869ad460; no voters:
I20251213 08:59:00.132159 20622 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56e271ad82a4526a70677cf90fb9705" candidate_uuid: "0b945a9ff9c4400b80f54337d0586e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" is_pre_election: true
I20251213 08:59:00.132473 20676 raft_consensus.cc:2804] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251213 08:59:00.132906 20676 raft_consensus.cc:493] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251213 08:59:00.132999 20622 raft_consensus.cc:2468] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b945a9ff9c4400b80f54337d0586e91 in term 0.
I20251213 08:59:00.133354 20676 raft_consensus.cc:3060] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:59:00.139981 20676 raft_consensus.cc:515] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:00.141856 20676 leader_election.cc:290] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [CANDIDATE]: Term 1 election: Requested vote from peers 3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379), 8e8c8b42dfa94db2860263cebcad3f7c (127.17.169.131:37829)
I20251213 08:59:00.142874 20548 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56e271ad82a4526a70677cf90fb9705" candidate_uuid: "0b945a9ff9c4400b80f54337d0586e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a18718f5cdd421e8960146e869ad460"
I20251213 08:59:00.143160 20622 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56e271ad82a4526a70677cf90fb9705" candidate_uuid: "0b945a9ff9c4400b80f54337d0586e91" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e8c8b42dfa94db2860263cebcad3f7c"
I20251213 08:59:00.143450 20548 raft_consensus.cc:3060] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:59:00.143735 20622 raft_consensus.cc:3060] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 0 FOLLOWER]: Advancing to term 1
W20251213 08:59:00.147637 20648 tablet.cc:2378] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251213 08:59:00.150647 20548 raft_consensus.cc:2468] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b945a9ff9c4400b80f54337d0586e91 in term 1.
I20251213 08:59:00.150918 20622 raft_consensus.cc:2468] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b945a9ff9c4400b80f54337d0586e91 in term 1.
I20251213 08:59:00.152000 20439 leader_election.cc:304] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b945a9ff9c4400b80f54337d0586e91, 3a18718f5cdd421e8960146e869ad460; no voters:
I20251213 08:59:00.152884 20676 raft_consensus.cc:2804] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:59:00.154129 20676 raft_consensus.cc:697] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 LEADER]: Becoming Leader. State: Replica: 0b945a9ff9c4400b80f54337d0586e91, State: Running, Role: LEADER
I20251213 08:59:00.155050 20676 consensus_queue.cc:237] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:00.165222 20368 catalog_manager.cc:5654] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b945a9ff9c4400b80f54337d0586e91 (127.17.169.129). New cstate: current_term: 1 leader_uuid: "0b945a9ff9c4400b80f54337d0586e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } health_report { overall_health: UNKNOWN } } }
I20251213 08:59:00.198463 18086 test_util.cc:276] Using random seed: -1765514224
I20251213 08:59:00.227962 18086 test_workload.cc:405] TestWorkload: Skipping table creation because table test-workload already exists
I20251213 08:59:00.321655 20622 raft_consensus.cc:1275] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 1 FOLLOWER]: Refusing update from remote peer 0b945a9ff9c4400b80f54337d0586e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:59:00.321928 20548 raft_consensus.cc:1275] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 1 FOLLOWER]: Refusing update from remote peer 0b945a9ff9c4400b80f54337d0586e91: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:59:00.323787 20676 consensus_queue.cc:1048] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251213 08:59:00.324957 20678 consensus_queue.cc:1048] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20251213 08:59:00.388554 20690 mvcc.cc:204] Tried to move back new op lower bound from 7231964529971847168 to 7231964529283866624. Current Snapshot: MvccSnapshot[applied={T|T < 7231964529971847168}]
I20251213 08:59:00.630298 20360 maintenance_manager.cc:419] P 815a02557c494c77a1e3fa3112b49991: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.003577
I20251213 08:59:00.775475 20359 maintenance_manager.cc:643] P 815a02557c494c77a1e3fa3112b49991: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.144s user 0.052s sys 0.033s Metrics: {"bytes_written":6637,"cfile_init":1,"dirs.queue_time_us":50136,"dirs.run_cpu_time_us":426,"dirs.run_wall_time_us":1077,"drs_written":1,"lbm_read_time_us":334,"lbm_reads_lt_1ms":4,"lbm_write_time_us":14683,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":5,"thread_start_us":50626,"threads_started":1}
I20251213 08:59:00.778656 20360 maintenance_manager.cc:419] P 815a02557c494c77a1e3fa3112b49991: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 489 bytes on disk
I20251213 08:59:00.780309 20359 maintenance_manager.cc:643] P 815a02557c494c77a1e3fa3112b49991: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":225,"lbm_reads_lt_1ms":4}
I20251213 08:59:01.152287 20648 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.242942
I20251213 08:59:01.181761 20499 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.242942
I20251213 08:59:01.292718 20574 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.242942
W20251213 08:59:01.962425 20688 meta_cache.cc:1261] Time spent looking up entry by key: real 0.052s user 0.004s sys 0.000s
I20251213 08:59:02.070734 20440 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.888s user 0.415s sys 0.028s Metrics: {"bytes_written":42202,"cfile_init":1,"dirs.queue_time_us":22444,"dirs.run_cpu_time_us":304,"dirs.run_wall_time_us":4998,"drs_written":1,"lbm_read_time_us":280,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2859,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1450,"thread_start_us":22699,"threads_started":1}
I20251213 08:59:02.073684 20499 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 12605 bytes on disk
I20251213 08:59:02.075390 20440 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":298,"lbm_reads_lt_1ms":4}
I20251213 08:59:02.092733 20589 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.940s user 0.433s sys 0.031s Metrics: {"bytes_written":40880,"cfile_init":1,"dirs.queue_time_us":35883,"dirs.run_cpu_time_us":288,"dirs.run_wall_time_us":3593,"drs_written":1,"lbm_read_time_us":281,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2848,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1400,"spinlock_wait_cycles":2432,"thread_start_us":35594,"threads_started":1}
I20251213 08:59:02.095711 20648 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 12190 bytes on disk
I20251213 08:59:02.097438 20589 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":293,"lbm_reads_lt_1ms":4}
I20251213 08:59:02.219524 20515 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.926s user 0.445s sys 0.034s Metrics: {"bytes_written":46048,"cfile_init":1,"dirs.queue_time_us":11202,"dirs.run_cpu_time_us":324,"dirs.run_wall_time_us":24428,"drs_written":1,"lbm_read_time_us":269,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2679,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1600,"spinlock_wait_cycles":258944,"thread_start_us":10893,"threads_started":1}
I20251213 08:59:02.222726 20574 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 13947 bytes on disk
I20251213 08:59:02.224421 20515 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":273,"lbm_reads_lt_1ms":4}
I20251213 08:59:03.084954 20499 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.486106
I20251213 08:59:03.103567 20648 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.486106
I20251213 08:59:03.239718 20574 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.486106
W20251213 08:59:03.571934 20686 meta_cache.cc:1261] Time spent looking up entry by key: real 0.051s user 0.002s sys 0.004s
I20251213 08:59:04.528501 20440 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.443s user 0.846s sys 0.023s Metrics: {"bytes_written":91195,"cfile_init":1,"dirs.queue_time_us":6848,"dirs.run_cpu_time_us":386,"dirs.run_wall_time_us":15165,"drs_written":1,"lbm_read_time_us":317,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3322,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3200,"spinlock_wait_cycles":222592,"thread_start_us":7108,"threads_started":1}
I20251213 08:59:04.535184 20499 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705): perf score=1.014500
I20251213 08:59:04.766737 20515 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.526s user 0.856s sys 0.009s Metrics: {"bytes_written":91243,"cfile_init":1,"dirs.queue_time_us":7070,"dirs.run_cpu_time_us":339,"dirs.run_wall_time_us":6715,"drs_written":1,"lbm_read_time_us":274,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3328,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3200,"spinlock_wait_cycles":208128,"thread_start_us":7318,"threads_started":1}
I20251213 08:59:04.770375 20574 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 28622 bytes on disk
I20251213 08:59:04.772197 20515 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":278,"lbm_reads_lt_1ms":4}
I20251213 08:59:04.774705 20574 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705): perf score=1.014512
I20251213 08:59:04.945353 20589 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.841s user 0.817s sys 0.028s Metrics: {"bytes_written":91152,"cfile_init":1,"dirs.queue_time_us":8985,"dirs.run_cpu_time_us":337,"dirs.run_wall_time_us":5564,"drs_written":1,"lbm_read_time_us":259,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3309,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3200,"spinlock_wait_cycles":237824,"thread_start_us":9198,"threads_started":1}
I20251213 08:59:04.949842 20648 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 28648 bytes on disk
I20251213 08:59:04.951685 20589 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":293,"lbm_reads_lt_1ms":4}
I20251213 08:59:04.956310 20648 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705): perf score=1.014495
I20251213 08:59:06.565336 20515 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.790s user 0.945s sys 0.055s Metrics: {"bytes_written":137043,"cfile_cache_hit":11,"cfile_cache_hit_bytes":93402,"cfile_cache_miss":27,"cfile_cache_miss_bytes":240034,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":20554,"dirs.run_cpu_time_us":1957,"dirs.run_wall_time_us":20096,"drs_written":1,"lbm_read_time_us":4349,"lbm_reads_lt_1ms":47,"lbm_write_time_us":4424,"lbm_writes_lt_1ms":27,"mutex_wait_us":761,"num_input_rowsets":2,"peak_mem_usage":75401,"rows_written":4800,"thread_start_us":94485,"threads_started":3}
I20251213 08:59:06.570528 20574 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.486106
I20251213 08:59:06.683984 20589 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.727s user 0.924s sys 0.046s Metrics: {"bytes_written":131716,"cfile_cache_hit":10,"cfile_cache_hit_bytes":84185,"cfile_cache_miss":28,"cfile_cache_miss_bytes":235271,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":12922,"dirs.run_cpu_time_us":320,"dirs.run_wall_time_us":2245,"drs_written":1,"lbm_read_time_us":4367,"lbm_reads_lt_1ms":48,"lbm_write_time_us":3930,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":72551,"rows_written":4600,"spinlock_wait_cycles":1920,"thread_start_us":12609,"threads_started":1}
I20251213 08:59:06.692271 20648 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.486106
I20251213 08:59:06.822455 20440 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 2.286s user 0.953s sys 0.020s Metrics: {"bytes_written":133181,"cfile_cache_hit":10,"cfile_cache_hit_bytes":85100,"cfile_cache_miss":28,"cfile_cache_miss_bytes":237904,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":51056,"dirs.run_cpu_time_us":1365,"dirs.run_wall_time_us":13250,"drs_written":1,"lbm_read_time_us":5237,"lbm_reads_lt_1ms":52,"lbm_write_time_us":4005,"lbm_writes_lt_1ms":27,"num_input_rowsets":2,"peak_mem_usage":71616,"rows_written":4650,"spinlock_wait_cycles":85679872,"thread_start_us":53721,"threads_started":4}
I20251213 08:59:06.833715 20499 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 42392 bytes on disk
I20251213 08:59:06.842890 20440 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":296,"lbm_reads_lt_1ms":4}
I20251213 08:59:06.848343 20499 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.486106
I20251213 08:59:07.877164 18086 tablet_server.cc:178] TabletServer@127.17.169.129:0 shutting down...
I20251213 08:59:08.317044 20515 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.746s user 0.919s sys 0.035s Metrics: {"bytes_written":110336,"cfile_init":1,"dirs.queue_time_us":4542,"dirs.run_cpu_time_us":346,"dirs.run_wall_time_us":1318,"drs_written":1,"lbm_read_time_us":211,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3103,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":3800,"thread_start_us":1282,"threads_started":1}
I20251213 08:59:08.319628 20574 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705): perf score=1.014851
W20251213 08:59:08.424412 20683 batcher.cc:441] Timed out: Failed to write batch of 50 ops to tablet a56e271ad82a4526a70677cf90fb9705 after 27 attempt(s): Failed to write to server: (no server available): Write(tablet: a56e271ad82a4526a70677cf90fb9705, num_ops: 50, num_attempts: 27) passed its deadline: Remote error: Service unavailable: service kudu.tserver.TabletServerService not registered on TabletServer
I20251213 08:59:08.459545 20589 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.766s user 1.155s sys 0.020s Metrics: {"bytes_written":119833,"cfile_init":1,"dirs.queue_time_us":3194,"dirs.run_cpu_time_us":364,"dirs.run_wall_time_us":4553,"drs_written":1,"lbm_read_time_us":256,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3775,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":4150,"spinlock_wait_cycles":16768,"thread_start_us":2824,"threads_started":1}
I20251213 08:59:08.463105 20648 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705): perf score=1.014845
I20251213 08:59:08.532428 20440 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.683s user 1.168s sys 0.012s Metrics: {"bytes_written":122451,"cfile_init":1,"dirs.queue_time_us":888,"dirs.run_cpu_time_us":328,"dirs.run_wall_time_us":1327,"drs_written":1,"lbm_read_time_us":245,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3619,"lbm_writes_lt_1ms":26,"peak_mem_usage":0,"rows_written":4250,"thread_start_us":543,"threads_started":1}
I20251213 08:59:08.534825 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:59:08.537843 18086 tablet_replica.cc:333] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: stopping tablet replica
I20251213 08:59:08.538640 18086 raft_consensus.cc:2243] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:59:08.540216 18086 raft_consensus.cc:2272] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251213 08:59:08.555172 20683 connection.cc:700] client connection to 127.17.169.129:40351 send error: Network error: sendmsg error: Broken pipe (error 32)
W20251213 08:59:08.555858 20683 meta_cache.cc:302] tablet a56e271ad82a4526a70677cf90fb9705: replica 0b945a9ff9c4400b80f54337d0586e91 (127.17.169.129:40351) has failed: Network error: sendmsg error: Broken pipe (error 32)
I20251213 08:59:08.583602 18086 tablet_server.cc:195] TabletServer@127.17.169.129:0 shutdown complete.
W20251213 08:59:08.619228 20528 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56218: Illegal state: replica 3a18718f5cdd421e8960146e869ad460 is not leader of this config: current role FOLLOWER
W20251213 08:59:08.620844 20525 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56218: Illegal state: replica 3a18718f5cdd421e8960146e869ad460 is not leader of this config: current role FOLLOWER
I20251213 08:59:08.619258 18086 tablet_server.cc:178] TabletServer@127.17.169.130:0 shutting down...
W20251213 08:59:08.623800 20527 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56218: Illegal state: replica 3a18718f5cdd421e8960146e869ad460 is not leader of this config: current role FOLLOWER
W20251213 08:59:08.625531 20526 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:56218: Illegal state: replica 3a18718f5cdd421e8960146e869ad460 is not leader of this config: current role FOLLOWER
W20251213 08:59:08.664709 20602 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41368: Illegal state: replica 8e8c8b42dfa94db2860263cebcad3f7c is not leader of this config: current role FOLLOWER
W20251213 08:59:08.667420 20602 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41368: Illegal state: replica 8e8c8b42dfa94db2860263cebcad3f7c is not leader of this config: current role FOLLOWER
W20251213 08:59:08.668293 20601 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41368: Illegal state: replica 8e8c8b42dfa94db2860263cebcad3f7c is not leader of this config: current role FOLLOWER
W20251213 08:59:08.671269 20601 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41368: Illegal state: replica 8e8c8b42dfa94db2860263cebcad3f7c is not leader of this config: current role FOLLOWER
W20251213 08:59:09.545266 20683 batcher.cc:441] Timed out: Failed to write batch of 50 ops to tablet a56e271ad82a4526a70677cf90fb9705 after 29 attempt(s): Failed to write to server: (no server available): Write(tablet: a56e271ad82a4526a70677cf90fb9705, num_ops: 50, num_attempts: 29) passed its deadline: Remote error: Service unavailable: service kudu.tserver.TabletServerService not registered on TabletServer
I20251213 08:59:09.614269 20515 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.294s user 1.244s sys 0.036s Metrics: {"bytes_written":249877,"cfile_cache_hit":10,"cfile_cache_hit_bytes":157936,"cfile_cache_miss":32,"cfile_cache_miss_bytes":441734,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":3677,"dirs.run_cpu_time_us":1972,"dirs.run_wall_time_us":13783,"drs_written":1,"lbm_read_time_us":6619,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":58,"lbm_write_time_us":4137,"lbm_writes_lt_1ms":31,"mutex_wait_us":79,"num_input_rowsets":2,"peak_mem_usage":132654,"rows_written":8600,"spinlock_wait_cycles":57088,"thread_start_us":7945,"threads_started":5,"wal-append.queue_time_us":896}
I20251213 08:59:09.616479 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:59:09.617290 18086 tablet_replica.cc:333] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: stopping tablet replica
I20251213 08:59:09.617856 18086 raft_consensus.cc:2243] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:59:09.618461 18086 raft_consensus.cc:2272] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251213 08:59:09.634657 20683 meta_cache.cc:302] tablet a56e271ad82a4526a70677cf90fb9705: replica 3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379) has failed: Network error: Client connection negotiation failed: client connection to 127.17.169.130:41379: connect: Connection refused (error 111)
W20251213 08:59:09.637145 20601 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41368: Illegal state: replica 8e8c8b42dfa94db2860263cebcad3f7c is not leader of this config: current role FOLLOWER
W20251213 08:59:09.639084 20602 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41368: Illegal state: replica 8e8c8b42dfa94db2860263cebcad3f7c is not leader of this config: current role FOLLOWER
I20251213 08:59:09.643762 18086 tablet_server.cc:195] TabletServer@127.17.169.130:0 shutdown complete.
W20251213 08:59:09.646525 20602 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41368: Illegal state: replica 8e8c8b42dfa94db2860263cebcad3f7c is not leader of this config: current role FOLLOWER
W20251213 08:59:09.655395 20602 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:41368: Illegal state: replica 8e8c8b42dfa94db2860263cebcad3f7c is not leader of this config: current role FOLLOWER
I20251213 08:59:09.665342 18086 tablet_server.cc:178] TabletServer@127.17.169.131:0 shutting down...
I20251213 08:59:09.804313 20589 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.334s user 1.275s sys 0.052s Metrics: {"bytes_written":253995,"cfile_cache_hit":10,"cfile_cache_hit_bytes":160532,"cfile_cache_miss":32,"cfile_cache_miss_bytes":450108,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":24252,"dirs.run_cpu_time_us":1221,"dirs.run_wall_time_us":5807,"drs_written":1,"lbm_read_time_us":7409,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":58,"lbm_write_time_us":4673,"lbm_writes_lt_1ms":31,"mutex_wait_us":9,"num_input_rowsets":2,"peak_mem_usage":133294,"rows_written":8750,"spinlock_wait_cycles":27008,"thread_start_us":24270,"threads_started":6,"wal-append.queue_time_us":678}
I20251213 08:59:09.806583 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:59:09.807178 18086 tablet_replica.cc:333] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: stopping tablet replica
I20251213 08:59:09.807672 18086 raft_consensus.cc:2243] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:59:09.808125 18086 raft_consensus.cc:2272] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:59:09.831560 18086 tablet_server.cc:195] TabletServer@127.17.169.131:0 shutdown complete.
I20251213 08:59:10.248237 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:59:10.255100 20764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:10.255120 20763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:10.256613 20766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:59:10.257797 18086 server_base.cc:1047] running on GCE node
I20251213 08:59:10.258709 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:59:10.258917 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:59:10.259071 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616350259051 us; error 0 us; skew 500 ppm
I20251213 08:59:10.259577 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:59:10.262070 18086 webserver.cc:492] Webserver started at http://127.17.169.129:46791/ using document root <none> and password file <none>
I20251213 08:59:10.262619 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:59:10.262810 18086 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251213 08:59:10.266710 18086 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20251213 08:59:10.280285 20774 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:10.281196 18086 fs_manager.cc:730] Time spent opening block manager: real 0.013s user 0.002s sys 0.001s
I20251213 08:59:10.281500 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-0-root
uuid: "0b945a9ff9c4400b80f54337d0586e91"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:59:10.281842 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-0-root/data
Total live blocks: 10
Total live bytes: 336319
Total live bytes (after alignment): 364544
Total number of LBM containers: 6 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:59:10.312163 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:59:10.313570 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:59:10.315078 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:59:10.318786 20781 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251213 08:59:10.325502 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251213 08:59:10.325781 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.008s user 0.000s sys 0.001s
I20251213 08:59:10.326056 18086 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251213 08:59:10.331549 18086 ts_tablet_manager.cc:616] Registered 1 tablets
I20251213 08:59:10.331770 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.006s user 0.005s sys 0.000s
I20251213 08:59:10.331977 20781 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: Bootstrap starting.
I20251213 08:59:10.383929 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.129:40351
I20251213 08:59:10.384020 20838 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.129:40351 every 8 connection(s)
I20251213 08:59:10.390142 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:59:10.404522 20843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:10.409528 20844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:10.422909 20846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:59:10.424005 18086 server_base.cc:1047] running on GCE node
I20251213 08:59:10.425390 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:59:10.425657 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:59:10.425869 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616350425852 us; error 0 us; skew 500 ppm
I20251213 08:59:10.426589 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:59:10.428268 20839 heartbeater.cc:344] Connected to a master server at 127.17.169.190:45943
I20251213 08:59:10.428648 20839 heartbeater.cc:461] Registering TS with master...
I20251213 08:59:10.429388 20839 heartbeater.cc:507] Master 127.17.169.190:45943 requested a full tablet report, sending...
I20251213 08:59:10.429627 18086 webserver.cc:492] Webserver started at http://127.17.169.130:37175/ using document root <none> and password file <none>
I20251213 08:59:10.430326 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:59:10.430547 18086 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251213 08:59:10.432125 20368 ts_manager.cc:194] Re-registered known tserver with Master: 0b945a9ff9c4400b80f54337d0586e91 (127.17.169.129:40351)
I20251213 08:59:10.435809 18086 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20251213 08:59:10.437409 20368 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58834
I20251213 08:59:10.453425 20853 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:10.454430 18086 fs_manager.cc:730] Time spent opening block manager: real 0.016s user 0.003s sys 0.001s
I20251213 08:59:10.454739 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-1-root
uuid: "3a18718f5cdd421e8960146e869ad460"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:59:10.455029 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-1-root/data
Total live blocks: 5
Total live bytes: 329624
Total live bytes (after alignment): 339968
Total number of LBM containers: 6 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:59:10.495002 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:59:10.496093 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:59:10.497706 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:59:10.508957 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251213 08:59:10.509195 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.009s user 0.001s sys 0.000s
I20251213 08:59:10.513439 18086 ts_tablet_manager.cc:616] Registered 1 tablets
I20251213 08:59:10.513720 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.001s sys 0.000s
I20251213 08:59:10.513783 20860 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: Bootstrap starting.
I20251213 08:59:10.577410 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.130:41379
I20251213 08:59:10.577620 20917 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.130:41379 every 8 connection(s)
I20251213 08:59:10.595880 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
I20251213 08:59:10.607721 20918 heartbeater.cc:344] Connected to a master server at 127.17.169.190:45943
I20251213 08:59:10.608244 20918 heartbeater.cc:461] Registering TS with master...
W20251213 08:59:10.609076 20921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:59:10.609233 20918 heartbeater.cc:507] Master 127.17.169.190:45943 requested a full tablet report, sending...
I20251213 08:59:10.612452 20368 ts_manager.cc:194] Re-registered known tserver with Master: 3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379)
I20251213 08:59:10.615382 20368 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58838
W20251213 08:59:10.618580 20922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:10.628954 20924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:59:10.631264 18086 server_base.cc:1047] running on GCE node
I20251213 08:59:10.636830 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:59:10.637107 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:59:10.637312 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616350637296 us; error 0 us; skew 500 ppm
I20251213 08:59:10.638036 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:59:10.641218 18086 webserver.cc:492] Webserver started at http://127.17.169.131:42649/ using document root <none> and password file <none>
I20251213 08:59:10.641852 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:59:10.642083 18086 fs_manager.cc:365] Using existing metadata directory in first data directory
I20251213 08:59:10.647279 18086 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20251213 08:59:10.671811 20932 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:10.672834 18086 fs_manager.cc:730] Time spent opening block manager: real 0.024s user 0.004s sys 0.000s
I20251213 08:59:10.673183 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-2-root
uuid: "8e8c8b42dfa94db2860263cebcad3f7c"
format_stamp: "Formatted at 2025-12-13 08:58:59 on dist-test-slave-s4vp"
I20251213 08:59:10.673542 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_0.1765616224759991-18086-0/minicluster-data/ts-2-root/data
Total live blocks: 5
Total live bytes: 335140
Total live bytes (after alignment): 344064
Total number of LBM containers: 6 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:59:10.746865 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:59:10.748633 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:59:10.750340 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:59:10.763476 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251213 08:59:10.763796 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.001s
I20251213 08:59:10.770119 18086 ts_tablet_manager.cc:616] Registered 1 tablets
I20251213 08:59:10.770485 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.006s user 0.005s sys 0.001s
I20251213 08:59:10.770496 20939 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Bootstrap starting.
I20251213 08:59:11.245869 20996 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.131:37829 every 8 connection(s)
I20251213 08:59:11.246387 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.131:37829
I20251213 08:59:11.282213 20997 heartbeater.cc:344] Connected to a master server at 127.17.169.190:45943
I20251213 08:59:11.282861 20997 heartbeater.cc:461] Registering TS with master...
I20251213 08:59:11.283820 20997 heartbeater.cc:507] Master 127.17.169.190:45943 requested a full tablet report, sending...
I20251213 08:59:11.286841 20368 ts_manager.cc:194] Re-registered known tserver with Master: 8e8c8b42dfa94db2860263cebcad3f7c (127.17.169.131:37829)
I20251213 08:59:11.289631 20368 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58848
I20251213 08:59:11.439816 20839 heartbeater.cc:499] Master 127.17.169.190:45943 was elected leader, sending a full tablet report...
I20251213 08:59:11.618453 20918 heartbeater.cc:499] Master 127.17.169.190:45943 was elected leader, sending a full tablet report...
I20251213 08:59:11.633919 20781 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: Bootstrap replayed 1/1 log segments. Stats: ops{read=204 overwritten=0 applied=204 ignored=178} inserts{seen=1250 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251213 08:59:11.635178 20781 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: Bootstrap complete.
I20251213 08:59:11.636066 20781 ts_tablet_manager.cc:1403] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: Time spent bootstrapping tablet: real 1.304s user 1.247s sys 0.036s
I20251213 08:59:11.638691 20781 raft_consensus.cc:359] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:11.639627 20781 raft_consensus.cc:740] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b945a9ff9c4400b80f54337d0586e91, State: Initialized, Role: FOLLOWER
I20251213 08:59:11.640398 20781 consensus_queue.cc:260] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 204, Last appended: 1.204, Last appended by leader: 204, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:11.654897 20781 ts_tablet_manager.cc:1434] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91: Time spent starting tablet: real 0.018s user 0.001s sys 0.016s
I20251213 08:59:11.773887 20804 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251213 08:59:11.808552 20883 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251213 08:59:11.816761 20962 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251213 08:59:11.828780 20860 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: Bootstrap replayed 1/1 log segments. Stats: ops{read=204 overwritten=0 applied=204 ignored=172} inserts{seen=1550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251213 08:59:11.830298 20860 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: Bootstrap complete.
I20251213 08:59:11.831106 20860 ts_tablet_manager.cc:1403] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: Time spent bootstrapping tablet: real 1.318s user 1.171s sys 0.036s
I20251213 08:59:11.833739 20860 raft_consensus.cc:359] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:11.834589 20860 raft_consensus.cc:740] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a18718f5cdd421e8960146e869ad460, State: Initialized, Role: FOLLOWER
I20251213 08:59:11.835454 20860 consensus_queue.cc:260] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 204, Last appended: 1.204, Last appended by leader: 204, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:11.867825 20860 ts_tablet_manager.cc:1434] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460: Time spent starting tablet: real 0.036s user 0.015s sys 0.020s
I20251213 08:59:11.900128 20840 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705): perf score=1.014843
I20251213 08:59:12.037696 20939 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Bootstrap replayed 1/1 log segments. Stats: ops{read=204 overwritten=0 applied=204 ignored=175} inserts{seen=1400 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251213 08:59:12.038910 20939 tablet_bootstrap.cc:492] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Bootstrap complete.
I20251213 08:59:12.039748 20939 ts_tablet_manager.cc:1403] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Time spent bootstrapping tablet: real 1.269s user 1.217s sys 0.024s
I20251213 08:59:12.042235 20939 raft_consensus.cc:359] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:12.043080 20939 raft_consensus.cc:740] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e8c8b42dfa94db2860263cebcad3f7c, State: Initialized, Role: FOLLOWER
I20251213 08:59:12.043970 20939 consensus_queue.cc:260] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 204, Last appended: 1.204, Last appended by leader: 204, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:12.057708 20997 heartbeater.cc:499] Master 127.17.169.190:45943 was elected leader, sending a full tablet report...
I20251213 08:59:12.058216 20939 ts_tablet_manager.cc:1434] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c: Time spent starting tablet: real 0.018s user 0.006s sys 0.012s
I20251213 08:59:12.097769 20919 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 79747 bytes on disk
I20251213 08:59:12.099849 20858 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":373,"lbm_reads_lt_1ms":4}
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
815a02557c494c77a1e3fa3112b49991 | 127.17.169.190:45943 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+----------------------+-------------------------
flush_threshold_mb | 1 | runtime,experimental | all 1 server(s) checked
flush_threshold_secs | 1 | runtime,experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
0b945a9ff9c4400b80f54337d0586e91 | 127.17.169.129:40351 | HEALTHY | <none> | 0 | 0
3a18718f5cdd421e8960146e869ad460 | 127.17.169.130:41379 | HEALTHY | <none> | 0 | 0
8e8c8b42dfa94db2860263cebcad3f7c | 127.17.169.131:37829 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+----------------------+-------------------------
flush_threshold_mb | 1 | runtime,experimental | all 3 server(s) checked
flush_threshold_secs | 1 | runtime,experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet a56e271ad82a4526a70677cf90fb9705 of table 'test-workload' is unavailable: 2 replica(s) not RUNNING
3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379): not running
State: BOOTSTRAPPING
Data state: TABLET_DATA_READY
Last status: Bootstrap starting.
0b945a9ff9c4400b80f54337d0586e91 (127.17.169.129:40351): RUNNING [LEADER]
8e8c8b42dfa94db2860263cebcad3f7c (127.17.169.131:37829): not running
State: BOOTSTRAPPING
Data state: TABLET_DATA_READY
Last status: Bootstrap starting.
All reported replicas are:
A = 3a18718f5cdd421e8960146e869ad460
B = 0b945a9ff9c4400b80f54337d0586e91
C = 8e8c8b42dfa94db2860263cebcad3f7c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+-------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | UNAVAILABLE | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20251213 08:59:12.120805 18086 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251213 08:59:12.277065 20998 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 81145 bytes on disk
I20251213 08:59:12.278924 20937 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":390,"lbm_reads_lt_1ms":4}
I20251213 08:59:12.514889 20804 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251213 08:59:12.520030 20883 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251213 08:59:12.522557 20962 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
815a02557c494c77a1e3fa3112b49991 | 127.17.169.190:45943 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+----------------------+-------------------------
flush_threshold_mb | 1 | runtime,experimental | all 1 server(s) checked
flush_threshold_secs | 1 | runtime,experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
0b945a9ff9c4400b80f54337d0586e91 | 127.17.169.129:40351 | HEALTHY | <none> | 0 | 0
3a18718f5cdd421e8960146e869ad460 | 127.17.169.130:41379 | HEALTHY | <none> | 0 | 0
8e8c8b42dfa94db2860263cebcad3f7c | 127.17.169.131:37829 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+----------------------+-------------------------
flush_threshold_mb | 1 | runtime,experimental | all 3 server(s) checked
flush_threshold_secs | 1 | runtime,experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet a56e271ad82a4526a70677cf90fb9705 of table 'test-workload' is conflicted: 3 replicas' active configs disagree with the leader master's
3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379): RUNNING
0b945a9ff9c4400b80f54337d0586e91 (127.17.169.129:40351): RUNNING [LEADER]
8e8c8b42dfa94db2860263cebcad3f7c (127.17.169.131:37829): RUNNING
All reported replicas are:
A = 3a18718f5cdd421e8960146e869ad460
B = 0b945a9ff9c4400b80f54337d0586e91
C = 8e8c8b42dfa94db2860263cebcad3f7c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 1 | -1 | Yes
B | A B C | 1 | -1 | Yes
C | A B C | 1 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20251213 08:59:12.776762 18086 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251213 08:59:13.094955 21063 raft_consensus.cc:493] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251213 08:59:13.095474 21063 raft_consensus.cc:515] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:13.097815 21063 leader_election.cc:290] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379), 8e8c8b42dfa94db2860263cebcad3f7c (127.17.169.131:37829)
I20251213 08:59:13.105463 20919 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.242942
I20251213 08:59:13.140313 20972 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56e271ad82a4526a70677cf90fb9705" candidate_uuid: "0b945a9ff9c4400b80f54337d0586e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 204 } } ignore_live_leader: false dest_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" is_pre_election: true
I20251213 08:59:13.141165 20972 raft_consensus.cc:2468] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b945a9ff9c4400b80f54337d0586e91 in term 1.
I20251213 08:59:13.143054 20778 leader_election.cc:304] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b945a9ff9c4400b80f54337d0586e91, 8e8c8b42dfa94db2860263cebcad3f7c; no voters:
I20251213 08:59:13.143163 20893 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56e271ad82a4526a70677cf90fb9705" candidate_uuid: "0b945a9ff9c4400b80f54337d0586e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 204 } } ignore_live_leader: false dest_uuid: "3a18718f5cdd421e8960146e869ad460" is_pre_election: true
I20251213 08:59:13.143852 20893 raft_consensus.cc:2468] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b945a9ff9c4400b80f54337d0586e91 in term 1.
I20251213 08:59:13.143929 21063 raft_consensus.cc:2804] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20251213 08:59:13.144340 21063 raft_consensus.cc:493] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251213 08:59:13.144655 21063 raft_consensus.cc:3060] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 1 FOLLOWER]: Advancing to term 2
I20251213 08:59:13.152567 21063 raft_consensus.cc:515] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:13.157249 20893 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56e271ad82a4526a70677cf90fb9705" candidate_uuid: "0b945a9ff9c4400b80f54337d0586e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 204 } } ignore_live_leader: false dest_uuid: "3a18718f5cdd421e8960146e869ad460"
I20251213 08:59:13.157850 20893 raft_consensus.cc:3060] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 1 FOLLOWER]: Advancing to term 2
I20251213 08:59:13.158073 20972 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a56e271ad82a4526a70677cf90fb9705" candidate_uuid: "0b945a9ff9c4400b80f54337d0586e91" candidate_term: 2 candidate_status { last_received { term: 1 index: 204 } } ignore_live_leader: false dest_uuid: "8e8c8b42dfa94db2860263cebcad3f7c"
I20251213 08:59:13.158650 20972 raft_consensus.cc:3060] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 1 FOLLOWER]: Advancing to term 2
I20251213 08:59:13.165396 20893 raft_consensus.cc:2468] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b945a9ff9c4400b80f54337d0586e91 in term 2.
I20251213 08:59:13.165556 20972 raft_consensus.cc:2468] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b945a9ff9c4400b80f54337d0586e91 in term 2.
I20251213 08:59:13.166246 21063 leader_election.cc:290] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [CANDIDATE]: Term 2 election: Requested vote from peers 3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379), 8e8c8b42dfa94db2860263cebcad3f7c (127.17.169.131:37829)
I20251213 08:59:13.166909 20778 leader_election.cc:304] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b945a9ff9c4400b80f54337d0586e91, 8e8c8b42dfa94db2860263cebcad3f7c; no voters:
I20251213 08:59:13.167716 21063 raft_consensus.cc:2804] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 2 FOLLOWER]: Leader election won for term 2
I20251213 08:59:13.181034 21063 raft_consensus.cc:697] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 2 LEADER]: Becoming Leader. State: Replica: 0b945a9ff9c4400b80f54337d0586e91, State: Running, Role: LEADER
I20251213 08:59:13.182092 21063 consensus_queue.cc:237] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 204, Committed index: 204, Last appended: 1.204, Last appended by leader: 204, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } }
I20251213 08:59:13.205096 20368 catalog_manager.cc:5654] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 reported cstate change: term changed from 1 to 2. New cstate: current_term: 2 leader_uuid: "0b945a9ff9c4400b80f54337d0586e91" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b945a9ff9c4400b80f54337d0586e91" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 40351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 } health_report { overall_health: UNKNOWN } } }
I20251213 08:59:13.263212 20779 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.362s user 1.239s sys 0.063s Metrics: {"bytes_written":257961,"cfile_cache_hit":10,"cfile_cache_hit_bytes":163422,"cfile_cache_miss":32,"cfile_cache_miss_bytes":458224,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":6723,"dirs.run_cpu_time_us":1950,"dirs.run_wall_time_us":7103,"drs_written":1,"lbm_read_time_us":12863,"lbm_reads_1-10_ms":3,"lbm_reads_lt_1ms":57,"lbm_write_time_us":4419,"lbm_writes_lt_1ms":32,"mutex_wait_us":6,"num_input_rowsets":2,"peak_mem_usage":133929,"rows_written":8900,"thread_start_us":9043,"threads_started":3}
I20251213 08:59:13.275105 20840 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 82837 bytes on disk
I20251213 08:59:13.277220 20779 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":361,"lbm_reads_lt_1ms":4}
I20251213 08:59:13.279495 20840 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.242942
I20251213 08:59:13.284993 20998 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling FlushMRSOp(a56e271ad82a4526a70677cf90fb9705): perf score=0.242942
I20251213 08:59:13.430774 20804 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251213 08:59:13.459329 20883 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251213 08:59:13.474020 20962 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251213 08:59:13.653823 20893 raft_consensus.cc:1275] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 2 FOLLOWER]: Refusing update from remote peer 0b945a9ff9c4400b80f54337d0586e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 204. Preceding OpId from leader: term: 2 index: 205. (index mismatch)
I20251213 08:59:13.655478 21063 consensus_queue.cc:1048] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a18718f5cdd421e8960146e869ad460" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 41379 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 205, Last known committed idx: 204, Time since last communication: 0.000s
I20251213 08:59:13.700681 20972 raft_consensus.cc:1275] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 2 FOLLOWER]: Refusing update from remote peer 0b945a9ff9c4400b80f54337d0586e91: Log matching property violated. Preceding OpId in replica: term: 1 index: 204. Preceding OpId from leader: term: 2 index: 205. (index mismatch)
I20251213 08:59:13.704516 21063 consensus_queue.cc:1048] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e8c8b42dfa94db2860263cebcad3f7c" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 37829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 205, Last known committed idx: 204, Time since last communication: 0.001s
I20251213 08:59:13.728546 20858 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.622s user 0.493s sys 0.048s Metrics: {"bytes_written":44693,"cfile_init":1,"dirs.queue_time_us":24853,"dirs.run_cpu_time_us":314,"dirs.run_wall_time_us":15399,"drs_written":1,"lbm_read_time_us":253,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2846,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1550,"thread_start_us":49538,"threads_started":2,"wal-append.queue_time_us":25360}
I20251213 08:59:13.732133 20919 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705): perf score=1.013006
I20251213 08:59:13.775878 20779 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.496s user 0.369s sys 0.008s Metrics: {"bytes_written":37099,"cfile_init":1,"dirs.queue_time_us":94,"dirs.run_cpu_time_us":239,"dirs.run_wall_time_us":808,"drs_written":1,"lbm_read_time_us":276,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2494,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1250}
I20251213 08:59:13.779484 20840 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705): perf score=1.011181
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
815a02557c494c77a1e3fa3112b49991 | 127.17.169.190:45943 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+----------------------+-------------------------
flush_threshold_mb | 1 | runtime,experimental | all 1 server(s) checked
flush_threshold_secs | 1 | runtime,experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
0b945a9ff9c4400b80f54337d0586e91 | 127.17.169.129:40351 | HEALTHY | <none> | 1 | 0
3a18718f5cdd421e8960146e869ad460 | 127.17.169.130:41379 | HEALTHY | <none> | 0 | 0
8e8c8b42dfa94db2860263cebcad3f7c | 127.17.169.131:37829 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+----------------------+-------------------------
flush_threshold_mb | 1 | runtime,experimental | all 3 server(s) checked
flush_threshold_secs | 1 | runtime,experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
Tablet a56e271ad82a4526a70677cf90fb9705 of table 'test-workload' is conflicted: 2 replicas' active configs disagree with the leader master's
3a18718f5cdd421e8960146e869ad460 (127.17.169.130:41379): RUNNING
0b945a9ff9c4400b80f54337d0586e91 (127.17.169.129:40351): RUNNING [LEADER]
8e8c8b42dfa94db2860263cebcad3f7c (127.17.169.131:37829): RUNNING
All reported replicas are:
A = 3a18718f5cdd421e8960146e869ad460
B = 0b945a9ff9c4400b80f54337d0586e91
C = 8e8c8b42dfa94db2860263cebcad3f7c
The consensus matrix is:
Config source | Replicas | Current term | Config index | Committed?
---------------+--------------+--------------+--------------+------------
master | A B* C | | | Yes
A | A B C | 2 | -1 | Yes
B | A B* C | 2 | -1 | Yes
C | A B C | 2 | -1 | Yes
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+--------------------+---------------+---------+------------+------------------+-------------
test-workload | 3 | CONSENSUS_MISMATCH | 1 | 0 | 0 | 0 | 1
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
==================
Errors:
==================
Corruption: table consistency check error: 1 out of 1 table(s) are not healthy
FAILED
I20251213 08:59:13.892792 18086 cluster_verifier.cc:83] Check not successful yet, sleeping and retrying: Runtime error: ksck discovered errors
I20251213 08:59:13.900975 20937 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: FlushMRSOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.615s user 0.391s sys 0.023s Metrics: {"bytes_written":40883,"cfile_init":1,"dirs.queue_time_us":10905,"dirs.run_cpu_time_us":290,"dirs.run_wall_time_us":4917,"drs_written":1,"lbm_read_time_us":259,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2819,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1400,"thread_start_us":10597,"threads_started":1}
I20251213 08:59:13.903383 20998 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705): perf score=1.011242
I20251213 08:59:14.342882 20360 maintenance_manager.cc:419] P 815a02557c494c77a1e3fa3112b49991: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.000728
I20251213 08:59:14.382134 20359 maintenance_manager.cc:643] P 815a02557c494c77a1e3fa3112b49991: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 0.038s user 0.019s sys 0.001s Metrics: {"bytes_written":430,"delete_count":0,"lbm_write_time_us":564,"lbm_writes_lt_1ms":4,"reinsert_count":0,"thread_start_us":15518,"threads_started":1,"update_count":1,"wal-append.queue_time_us":15254}
I20251213 08:59:14.384497 20360 maintenance_manager.cc:419] P 815a02557c494c77a1e3fa3112b49991: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.196886
I20251213 08:59:14.471920 20359 maintenance_manager.cc:643] P 815a02557c494c77a1e3fa3112b49991: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.086s user 0.033s sys 0.002s Metrics: {"cfile_cache_miss":5,"cfile_cache_miss_bytes":2100,"cfile_init":4,"delete_count":0,"delta_blocks_compacted":1,"delta_iterators_relevant":1,"dirs.queue_time_us":20702,"dirs.run_cpu_time_us":510,"dirs.run_wall_time_us":1743,"lbm_read_time_us":1650,"lbm_reads_lt_1ms":21,"lbm_write_time_us":1076,"lbm_writes_lt_1ms":9,"peak_mem_usage":32768,"reinsert_count":0,"thread_start_us":24802,"threads_started":2,"update_count":1}
I20251213 08:59:14.790724 20804 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251213 08:59:14.829041 20883 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251213 08:59:14.892125 20962 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
Master Summary
UUID | Address | Status
----------------------------------+----------------------+---------
815a02557c494c77a1e3fa3112b49991 | 127.17.169.190:45943 | HEALTHY
Unusual flags for Master:
Flag | Value | Tags | Master
----------------------------------+-------+----------------------+-------------------------
flush_threshold_mb | 1 | runtime,experimental | all 1 server(s) checked
flush_threshold_secs | 1 | runtime,experimental | all 1 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 1 server(s) checked
Flags of checked categories for Master:
Flag | Value | Master
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 1 server(s) checked
time_source | system_unsync | all 1 server(s) checked
Tablet Server Summary
UUID | Address | Status | Location | Tablet Leaders | Active Scanners
----------------------------------+----------------------+---------+----------+----------------+-----------------
0b945a9ff9c4400b80f54337d0586e91 | 127.17.169.129:40351 | HEALTHY | <none> | 1 | 0
3a18718f5cdd421e8960146e869ad460 | 127.17.169.130:41379 | HEALTHY | <none> | 0 | 0
8e8c8b42dfa94db2860263cebcad3f7c | 127.17.169.131:37829 | HEALTHY | <none> | 0 | 0
Tablet Server Location Summary
Location | Count
----------+---------
<none> | 3
Unusual flags for Tablet Server:
Flag | Value | Tags | Tablet Server
----------------------------------+-------+----------------------+-------------------------
flush_threshold_mb | 1 | runtime,experimental | all 3 server(s) checked
flush_threshold_secs | 1 | runtime,experimental | all 3 server(s) checked
rpc_server_allow_ephemeral_ports | true | unsafe | all 3 server(s) checked
Flags of checked categories for Tablet Server:
Flag | Value | Tablet Server
---------------------+-------------------------------------------------------------+-------------------------
builtin_ntp_servers | 0.pool.ntp.org,1.pool.ntp.org,2.pool.ntp.org,3.pool.ntp.org | all 3 server(s) checked
time_source | system_unsync | all 3 server(s) checked
Version Summary
Version | Servers
-----------------+-------------------------
1.19.0-SNAPSHOT | all 4 server(s) checked
Tablet Summary
The cluster doesn't have any matching system tables
Summary by table
Name | RF | Status | Total Tablets | Healthy | Recovering | Under-replicated | Unavailable
---------------+----+---------+---------------+---------+------------+------------------+-------------
test-workload | 3 | HEALTHY | 1 | 1 | 0 | 0 | 0
Tablet Replica Count Summary
Statistic | Replica Count
----------------+---------------
Minimum | 1
First Quartile | 1
Median | 1
Third Quartile | 1
Maximum | 1
Total Count Summary
| Total Count
----------------+-------------
Masters | 1
Tablet Servers | 3
Tables | 1
Tablets | 1
Replicas | 3
==================
Warnings:
==================
Some masters have unsafe, experimental, or hidden flags set
Some tablet servers have unsafe, experimental, or hidden flags set
OK
I20251213 08:59:15.257673 18086 log_verifier.cc:126] Checking tablet a56e271ad82a4526a70677cf90fb9705
I20251213 08:59:15.874150 20937 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 1.970s user 1.822s sys 0.080s Metrics: {"bytes_written":290915,"cfile_cache_hit":10,"cfile_cache_hit_bytes":190178,"cfile_cache_miss":32,"cfile_cache_miss_bytes":527080,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":10453,"dirs.run_cpu_time_us":1491,"dirs.run_wall_time_us":21949,"drs_written":1,"lbm_read_time_us":6561,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":6152,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":31,"num_input_rowsets":2,"peak_mem_usage":155532,"rows_written":10150,"thread_start_us":26654,"threads_started":4,"wal-append.queue_time_us":6854}
I20251213 08:59:15.877326 20998 maintenance_manager.cc:419] P 8e8c8b42dfa94db2860263cebcad3f7c: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 94937 bytes on disk
I20251213 08:59:15.879001 20937 maintenance_manager.cc:643] P 8e8c8b42dfa94db2860263cebcad3f7c: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":268,"lbm_reads_lt_1ms":4}
I20251213 08:59:15.941031 20779 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 2.161s user 2.020s sys 0.044s Metrics: {"bytes_written":290915,"cfile_cache_hit":10,"cfile_cache_hit_bytes":190350,"cfile_cache_miss":34,"cfile_cache_miss_bytes":528038,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":18576,"dirs.run_cpu_time_us":1661,"dirs.run_wall_time_us":8725,"drs_written":1,"lbm_read_time_us":8111,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":57,"lbm_write_time_us":5953,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":31,"mutex_wait_us":109,"num_input_rowsets":2,"peak_mem_usage":155680,"rows_written":10150,"spinlock_wait_cycles":2816,"thread_start_us":20061,"threads_started":5,"wal-append.queue_time_us":1089}
I20251213 08:59:15.945030 20840 maintenance_manager.cc:419] P 0b945a9ff9c4400b80f54337d0586e91: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 94937 bytes on disk
I20251213 08:59:15.946877 20779 maintenance_manager.cc:643] P 0b945a9ff9c4400b80f54337d0586e91: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":356,"lbm_reads_lt_1ms":4}
I20251213 08:59:16.027519 20858 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: CompactRowSetsOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 2.295s user 1.908s sys 0.026s Metrics: {"bytes_written":290915,"cfile_cache_hit":10,"cfile_cache_hit_bytes":190028,"cfile_cache_miss":32,"cfile_cache_miss_bytes":526540,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":9440,"dirs.run_cpu_time_us":1261,"dirs.run_wall_time_us":9444,"drs_written":1,"lbm_read_time_us":8728,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":55,"lbm_write_time_us":5698,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":31,"mutex_wait_us":131,"num_input_rowsets":2,"peak_mem_usage":155382,"rows_written":10150,"spinlock_wait_cycles":9984,"thread_start_us":10499,"threads_started":5,"wal-append.queue_time_us":1505}
I20251213 08:59:16.030551 20919 maintenance_manager.cc:419] P 3a18718f5cdd421e8960146e869ad460: Scheduling UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705): 94937 bytes on disk
I20251213 08:59:16.032209 20858 maintenance_manager.cc:643] P 3a18718f5cdd421e8960146e869ad460: UndoDeltaBlockGCOp(a56e271ad82a4526a70677cf90fb9705) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":230,"lbm_reads_lt_1ms":4}
I20251213 08:59:16.334615 18086 log_verifier.cc:177] Verified matching terms for 205 ops in tablet a56e271ad82a4526a70677cf90fb9705
I20251213 08:59:16.337059 18086 tablet_server.cc:178] TabletServer@127.17.169.129:40351 shutting down...
I20251213 08:59:16.365540 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:59:16.366148 18086 tablet_replica.cc:333] stopping tablet replica
I20251213 08:59:16.366710 18086 raft_consensus.cc:2243] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 2 LEADER]: Raft consensus shutting down.
I20251213 08:59:16.367460 18086 raft_consensus.cc:2272] T a56e271ad82a4526a70677cf90fb9705 P 0b945a9ff9c4400b80f54337d0586e91 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251213 08:59:16.387235 18086 tablet_server.cc:195] TabletServer@127.17.169.129:40351 shutdown complete.
I20251213 08:59:16.398785 18086 tablet_server.cc:178] TabletServer@127.17.169.130:41379 shutting down...
I20251213 08:59:16.416707 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:59:16.417239 18086 tablet_replica.cc:333] stopping tablet replica
I20251213 08:59:16.417690 18086 raft_consensus.cc:2243] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 2 FOLLOWER]: Raft consensus shutting down.
I20251213 08:59:16.418046 18086 raft_consensus.cc:2272] T a56e271ad82a4526a70677cf90fb9705 P 3a18718f5cdd421e8960146e869ad460 [term 2 FOLLOWER]: Raft consensus is shut down!
I20251213 08:59:16.436901 18086 tablet_server.cc:195] TabletServer@127.17.169.130:41379 shutdown complete.
I20251213 08:59:16.448819 18086 tablet_server.cc:178] TabletServer@127.17.169.131:37829 shutting down...
I20251213 08:59:16.465617 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:59:16.466234 18086 tablet_replica.cc:333] stopping tablet replica
I20251213 08:59:16.466715 18086 raft_consensus.cc:2243] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 2 FOLLOWER]: Raft consensus shutting down.
I20251213 08:59:16.467110 18086 raft_consensus.cc:2272] T a56e271ad82a4526a70677cf90fb9705 P 8e8c8b42dfa94db2860263cebcad3f7c [term 2 FOLLOWER]: Raft consensus is shut down!
I20251213 08:59:16.485981 18086 tablet_server.cc:195] TabletServer@127.17.169.131:37829 shutdown complete.
I20251213 08:59:16.524806 18086 master.cc:561] Master@127.17.169.190:45943 shutting down...
I20251213 08:59:16.547086 18086 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 1 LEADER]: Raft consensus shutting down.
I20251213 08:59:16.548110 18086 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991 [term 1 FOLLOWER]: Raft consensus is shut down!
I20251213 08:59:16.548487 18086 tablet_replica.cc:333] T 00000000000000000000000000000000 P 815a02557c494c77a1e3fa3112b49991: stopping tablet replica
I20251213 08:59:16.569983 18086 master.cc:583] Master@127.17.169.190:45943 shutdown complete.
[ OK ] StopTablets/StopTabletITest.TestShutdownWhileWriting/0 (17297 ms)
[ RUN ] StopTablets/StopTabletITest.TestShutdownWhileWriting/1
I20251213 08:59:16.614022 18086 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.17.169.190:35729
I20251213 08:59:16.615180 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:59:16.621255 21151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:16.621817 21153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:16.621955 21150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:59:16.622349 18086 server_base.cc:1047] running on GCE node
I20251213 08:59:16.624065 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:59:16.624277 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:59:16.624428 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616356624410 us; error 0 us; skew 500 ppm
I20251213 08:59:16.624953 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:59:16.627336 18086 webserver.cc:492] Webserver started at http://127.17.169.190:41827/ using document root <none> and password file <none>
I20251213 08:59:16.627833 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:59:16.628017 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:59:16.628263 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:59:16.629479 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/master-0-root/instance:
uuid: "b1219ee1e83c483795aa5c25fa3d1d14"
format_stamp: "Formatted at 2025-12-13 08:59:16 on dist-test-slave-s4vp"
I20251213 08:59:16.633874 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20251213 08:59:16.637159 21158 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:16.637841 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251213 08:59:16.638082 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/master-0-root
uuid: "b1219ee1e83c483795aa5c25fa3d1d14"
format_stamp: "Formatted at 2025-12-13 08:59:16 on dist-test-slave-s4vp"
I20251213 08:59:16.638360 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:59:16.667528 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:59:16.668768 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:59:16.707302 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.190:35729
I20251213 08:59:16.707437 21209 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.190:35729 every 8 connection(s)
I20251213 08:59:16.711561 21210 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:59:16.723053 21210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14: Bootstrap starting.
I20251213 08:59:16.727775 21210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14: Neither blocks nor log segments found. Creating new log.
I20251213 08:59:16.731950 21210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14: No bootstrap required, opened a new log
I20251213 08:59:16.734076 21210 raft_consensus.cc:359] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1219ee1e83c483795aa5c25fa3d1d14" member_type: VOTER }
I20251213 08:59:16.734571 21210 raft_consensus.cc:385] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:59:16.734797 21210 raft_consensus.cc:740] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1219ee1e83c483795aa5c25fa3d1d14, State: Initialized, Role: FOLLOWER
I20251213 08:59:16.735380 21210 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1219ee1e83c483795aa5c25fa3d1d14" member_type: VOTER }
I20251213 08:59:16.735823 21210 raft_consensus.cc:399] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251213 08:59:16.736029 21210 raft_consensus.cc:493] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251213 08:59:16.736289 21210 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:59:16.741168 21210 raft_consensus.cc:515] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1219ee1e83c483795aa5c25fa3d1d14" member_type: VOTER }
I20251213 08:59:16.741707 21210 leader_election.cc:304] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1219ee1e83c483795aa5c25fa3d1d14; no voters:
I20251213 08:59:16.742864 21210 leader_election.cc:290] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251213 08:59:16.743168 21213 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:59:16.744477 21213 raft_consensus.cc:697] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [term 1 LEADER]: Becoming Leader. State: Replica: b1219ee1e83c483795aa5c25fa3d1d14, State: Running, Role: LEADER
I20251213 08:59:16.745169 21213 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1219ee1e83c483795aa5c25fa3d1d14" member_type: VOTER }
I20251213 08:59:16.745849 21210 sys_catalog.cc:565] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [sys.catalog]: configured and running, proceeding with master startup.
I20251213 08:59:16.751406 21214 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1219ee1e83c483795aa5c25fa3d1d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1219ee1e83c483795aa5c25fa3d1d14" member_type: VOTER } }
I20251213 08:59:16.751502 21215 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1219ee1e83c483795aa5c25fa3d1d14. Latest consensus state: current_term: 1 leader_uuid: "b1219ee1e83c483795aa5c25fa3d1d14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1219ee1e83c483795aa5c25fa3d1d14" member_type: VOTER } }
I20251213 08:59:16.752503 21215 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [sys.catalog]: This master's current role is: LEADER
I20251213 08:59:16.752460 21214 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14 [sys.catalog]: This master's current role is: LEADER
I20251213 08:59:16.758538 21221 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251213 08:59:16.763325 21221 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251213 08:59:16.764281 18086 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20251213 08:59:16.771734 21221 catalog_manager.cc:1357] Generated new cluster ID: d539deae042a49adbbb45220883b057f
I20251213 08:59:16.772028 21221 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251213 08:59:16.786507 21221 catalog_manager.cc:1380] Generated new certificate authority record
I20251213 08:59:16.787793 21221 catalog_manager.cc:1514] Loading token signing keys...
I20251213 08:59:16.802336 21221 catalog_manager.cc:6027] T 00000000000000000000000000000000 P b1219ee1e83c483795aa5c25fa3d1d14: Generated new TSK 0
I20251213 08:59:16.802897 21221 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251213 08:59:16.830895 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:59:16.836901 21231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:16.838326 21232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:16.839305 21234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:59:16.839656 18086 server_base.cc:1047] running on GCE node
I20251213 08:59:16.840545 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:59:16.840754 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:59:16.840931 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616356840912 us; error 0 us; skew 500 ppm
I20251213 08:59:16.841437 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:59:16.843834 18086 webserver.cc:492] Webserver started at http://127.17.169.129:36171/ using document root <none> and password file <none>
I20251213 08:59:16.844293 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:59:16.844485 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:59:16.844731 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:59:16.845883 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-0-root/instance:
uuid: "1646c503014749619079a5e1d3b9d033"
format_stamp: "Formatted at 2025-12-13 08:59:16 on dist-test-slave-s4vp"
I20251213 08:59:16.850303 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20251213 08:59:16.853614 21239 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:16.854408 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251213 08:59:16.854650 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-0-root
uuid: "1646c503014749619079a5e1d3b9d033"
format_stamp: "Formatted at 2025-12-13 08:59:16 on dist-test-slave-s4vp"
I20251213 08:59:16.854887 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:59:16.876995 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:59:16.878129 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:59:16.879583 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:59:16.881793 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:59:16.881978 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:16.882244 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:59:16.882423 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:16.923605 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.129:36931
I20251213 08:59:16.923720 21301 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.129:36931 every 8 connection(s)
I20251213 08:59:16.928321 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:59:16.936733 21306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:16.938295 21307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:16.941273 21309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:59:16.941715 18086 server_base.cc:1047] running on GCE node
I20251213 08:59:16.942513 21302 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35729
I20251213 08:59:16.942824 21302 heartbeater.cc:461] Registering TS with master...
I20251213 08:59:16.942898 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:59:16.943148 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:59:16.943312 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616356943293 us; error 0 us; skew 500 ppm
I20251213 08:59:16.943552 21302 heartbeater.cc:507] Master 127.17.169.190:35729 requested a full tablet report, sending...
I20251213 08:59:16.943890 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:59:16.945801 21175 ts_manager.cc:194] Registered new tserver with Master: 1646c503014749619079a5e1d3b9d033 (127.17.169.129:36931)
I20251213 08:59:16.946748 18086 webserver.cc:492] Webserver started at http://127.17.169.130:42633/ using document root <none> and password file <none>
I20251213 08:59:16.947506 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:59:16.947774 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:59:16.948069 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:59:16.948388 21175 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54220
I20251213 08:59:16.949666 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-1-root/instance:
uuid: "48f1c753536c4189a1f7900702aff671"
format_stamp: "Formatted at 2025-12-13 08:59:16 on dist-test-slave-s4vp"
I20251213 08:59:16.954022 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20251213 08:59:16.957408 21314 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:16.958289 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20251213 08:59:16.958559 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-1-root
uuid: "48f1c753536c4189a1f7900702aff671"
format_stamp: "Formatted at 2025-12-13 08:59:16 on dist-test-slave-s4vp"
I20251213 08:59:16.958812 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:59:16.973237 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:59:16.974438 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:59:16.975850 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:59:16.978004 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:59:16.978236 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:16.978487 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:59:16.978631 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:17.018633 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.130:37571
I20251213 08:59:17.018733 21376 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.130:37571 every 8 connection(s)
I20251213 08:59:17.023485 18086 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251213 08:59:17.031651 21380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:17.032756 21381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251213 08:59:17.034493 21383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251213 08:59:17.034950 21377 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35729
I20251213 08:59:17.035282 21377 heartbeater.cc:461] Registering TS with master...
I20251213 08:59:17.035847 18086 server_base.cc:1047] running on GCE node
I20251213 08:59:17.035971 21377 heartbeater.cc:507] Master 127.17.169.190:35729 requested a full tablet report, sending...
I20251213 08:59:17.036746 18086 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20251213 08:59:17.036944 18086 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20251213 08:59:17.037122 18086 hybrid_clock.cc:648] HybridClock initialized: now 1765616357037102 us; error 0 us; skew 500 ppm
I20251213 08:59:17.037835 18086 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251213 08:59:17.037979 21175 ts_manager.cc:194] Registered new tserver with Master: 48f1c753536c4189a1f7900702aff671 (127.17.169.130:37571)
I20251213 08:59:17.039808 21175 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54226
I20251213 08:59:17.040592 18086 webserver.cc:492] Webserver started at http://127.17.169.131:44473/ using document root <none> and password file <none>
I20251213 08:59:17.041142 18086 fs_manager.cc:362] Metadata directory not provided
I20251213 08:59:17.041358 18086 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251213 08:59:17.041597 18086 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251213 08:59:17.042742 18086 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-2-root/instance:
uuid: "53b3a70556a64f3f9720cc0d6cecfb0a"
format_stamp: "Formatted at 2025-12-13 08:59:17 on dist-test-slave-s4vp"
I20251213 08:59:17.047127 18086 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20251213 08:59:17.050622 21388 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:17.051460 18086 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20251213 08:59:17.051712 18086 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-2-root
uuid: "53b3a70556a64f3f9720cc0d6cecfb0a"
format_stamp: "Formatted at 2025-12-13 08:59:17 on dist-test-slave-s4vp"
I20251213 08:59:17.051941 18086 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskowm870/test-tmp/stop_tablet-itest.0.StopTablets_StopTabletITest.TestShutdownWhileWriting_1.1765616224759991-18086-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251213 08:59:17.064143 18086 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251213 08:59:17.065238 18086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251213 08:59:17.066637 18086 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251213 08:59:17.068811 18086 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251213 08:59:17.069016 18086 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:17.069242 18086 ts_tablet_manager.cc:616] Registered 0 tablets
I20251213 08:59:17.069394 18086 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251213 08:59:17.113121 18086 rpc_server.cc:307] RPC server started. Bound to: 127.17.169.131:43595
I20251213 08:59:17.113262 21450 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.17.169.131:43595 every 8 connection(s)
I20251213 08:59:17.126602 21451 heartbeater.cc:344] Connected to a master server at 127.17.169.190:35729
I20251213 08:59:17.126930 21451 heartbeater.cc:461] Registering TS with master...
I20251213 08:59:17.127653 21451 heartbeater.cc:507] Master 127.17.169.190:35729 requested a full tablet report, sending...
I20251213 08:59:17.129390 21175 ts_manager.cc:194] Registered new tserver with Master: 53b3a70556a64f3f9720cc0d6cecfb0a (127.17.169.131:43595)
I20251213 08:59:17.129824 18086 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013367302s
I20251213 08:59:17.130764 21175 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:54238
I20251213 08:59:17.163625 18086 test_util.cc:276] Using random seed: -1748549038
I20251213 08:59:17.185336 21175 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54268:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251213 08:59:17.187299 21175 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251213 08:59:17.236429 21267 tablet_service.cc:1505] Processing CreateTablet for tablet d06ff3930e7944b2bab03e288821506a (DEFAULT_TABLE table=test-workload [id=ba04e1e0cfe541e7b1f419aa9cb66835]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:59:17.236644 21416 tablet_service.cc:1505] Processing CreateTablet for tablet d06ff3930e7944b2bab03e288821506a (DEFAULT_TABLE table=test-workload [id=ba04e1e0cfe541e7b1f419aa9cb66835]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:59:17.237860 21416 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06ff3930e7944b2bab03e288821506a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:59:17.240948 21342 tablet_service.cc:1505] Processing CreateTablet for tablet d06ff3930e7944b2bab03e288821506a (DEFAULT_TABLE table=test-workload [id=ba04e1e0cfe541e7b1f419aa9cb66835]), partition=RANGE (key) PARTITION UNBOUNDED
I20251213 08:59:17.241868 21342 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06ff3930e7944b2bab03e288821506a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:59:17.237846 21267 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d06ff3930e7944b2bab03e288821506a. 1 dirs total, 0 dirs full, 0 dirs failed
I20251213 08:59:17.255105 21472 tablet_bootstrap.cc:492] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a: Bootstrap starting.
I20251213 08:59:17.259871 21474 tablet_bootstrap.cc:492] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033: Bootstrap starting.
I20251213 08:59:17.261473 21473 tablet_bootstrap.cc:492] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671: Bootstrap starting.
I20251213 08:59:17.266088 21474 tablet_bootstrap.cc:654] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033: Neither blocks nor log segments found. Creating new log.
I20251213 08:59:17.267714 21472 tablet_bootstrap.cc:654] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a: Neither blocks nor log segments found. Creating new log.
I20251213 08:59:17.268718 21473 tablet_bootstrap.cc:654] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671: Neither blocks nor log segments found. Creating new log.
I20251213 08:59:17.271761 21474 tablet_bootstrap.cc:492] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033: No bootstrap required, opened a new log
I20251213 08:59:17.272248 21474 ts_tablet_manager.cc:1403] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20251213 08:59:17.273154 21472 tablet_bootstrap.cc:492] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a: No bootstrap required, opened a new log
I20251213 08:59:17.273620 21472 ts_tablet_manager.cc:1403] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a: Time spent bootstrapping tablet: real 0.019s user 0.012s sys 0.000s
I20251213 08:59:17.273979 21473 tablet_bootstrap.cc:492] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671: No bootstrap required, opened a new log
I20251213 08:59:17.274485 21473 ts_tablet_manager.cc:1403] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671: Time spent bootstrapping tablet: real 0.013s user 0.013s sys 0.000s
I20251213 08:59:17.274492 21474 raft_consensus.cc:359] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:17.275189 21474 raft_consensus.cc:385] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:59:17.275445 21474 raft_consensus.cc:740] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1646c503014749619079a5e1d3b9d033, State: Initialized, Role: FOLLOWER
I20251213 08:59:17.276019 21474 consensus_queue.cc:260] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:17.276386 21472 raft_consensus.cc:359] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:17.277158 21472 raft_consensus.cc:385] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:59:17.277539 21472 raft_consensus.cc:740] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53b3a70556a64f3f9720cc0d6cecfb0a, State: Initialized, Role: FOLLOWER
I20251213 08:59:17.277233 21473 raft_consensus.cc:359] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:17.277993 21473 raft_consensus.cc:385] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251213 08:59:17.278323 21473 raft_consensus.cc:740] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48f1c753536c4189a1f7900702aff671, State: Initialized, Role: FOLLOWER
I20251213 08:59:17.278288 21472 consensus_queue.cc:260] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:17.279254 21473 consensus_queue.cc:260] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:17.282182 21377 heartbeater.cc:499] Master 127.17.169.190:35729 was elected leader, sending a full tablet report...
I20251213 08:59:17.283150 21473 ts_tablet_manager.cc:1434] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20251213 08:59:17.284965 21451 heartbeater.cc:499] Master 127.17.169.190:35729 was elected leader, sending a full tablet report...
I20251213 08:59:17.285668 21472 ts_tablet_manager.cc:1434] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a: Time spent starting tablet: real 0.012s user 0.000s sys 0.013s
I20251213 08:59:17.286760 21302 heartbeater.cc:499] Master 127.17.169.190:35729 was elected leader, sending a full tablet report...
I20251213 08:59:17.287209 21474 ts_tablet_manager.cc:1434] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20251213 08:59:17.476908 21479 raft_consensus.cc:493] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251213 08:59:17.477339 21479 raft_consensus.cc:515] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:17.479388 21479 leader_election.cc:290] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1646c503014749619079a5e1d3b9d033 (127.17.169.129:36931), 48f1c753536c4189a1f7900702aff671 (127.17.169.130:37571)
I20251213 08:59:17.488826 21277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06ff3930e7944b2bab03e288821506a" candidate_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1646c503014749619079a5e1d3b9d033" is_pre_election: true
I20251213 08:59:17.489620 21277 raft_consensus.cc:2468] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53b3a70556a64f3f9720cc0d6cecfb0a in term 0.
I20251213 08:59:17.490134 21352 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06ff3930e7944b2bab03e288821506a" candidate_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f1c753536c4189a1f7900702aff671" is_pre_election: true
I20251213 08:59:17.490808 21352 raft_consensus.cc:2468] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53b3a70556a64f3f9720cc0d6cecfb0a in term 0.
I20251213 08:59:17.490767 21389 leader_election.cc:304] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1646c503014749619079a5e1d3b9d033, 53b3a70556a64f3f9720cc0d6cecfb0a; no voters:
I20251213 08:59:17.491725 21479 raft_consensus.cc:2804] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251213 08:59:17.492091 21479 raft_consensus.cc:493] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251213 08:59:17.492539 21479 raft_consensus.cc:3060] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:59:17.497591 21479 raft_consensus.cc:515] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:17.499070 21479 leader_election.cc:290] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [CANDIDATE]: Term 1 election: Requested vote from peers 1646c503014749619079a5e1d3b9d033 (127.17.169.129:36931), 48f1c753536c4189a1f7900702aff671 (127.17.169.130:37571)
I20251213 08:59:17.499969 21277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06ff3930e7944b2bab03e288821506a" candidate_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1646c503014749619079a5e1d3b9d033"
I20251213 08:59:17.500250 21352 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06ff3930e7944b2bab03e288821506a" candidate_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48f1c753536c4189a1f7900702aff671"
I20251213 08:59:17.500430 21277 raft_consensus.cc:3060] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:59:17.500782 21352 raft_consensus.cc:3060] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671 [term 0 FOLLOWER]: Advancing to term 1
I20251213 08:59:17.505733 21277 raft_consensus.cc:2468] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53b3a70556a64f3f9720cc0d6cecfb0a in term 1.
I20251213 08:59:17.505810 21352 raft_consensus.cc:2468] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53b3a70556a64f3f9720cc0d6cecfb0a in term 1.
I20251213 08:59:17.506735 21389 leader_election.cc:304] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1646c503014749619079a5e1d3b9d033, 53b3a70556a64f3f9720cc0d6cecfb0a; no voters:
I20251213 08:59:17.507382 21479 raft_consensus.cc:2804] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 1 FOLLOWER]: Leader election won for term 1
I20251213 08:59:17.508312 21479 raft_consensus.cc:697] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [term 1 LEADER]: Becoming Leader. State: Replica: 53b3a70556a64f3f9720cc0d6cecfb0a, State: Running, Role: LEADER
I20251213 08:59:17.509064 21479 consensus_queue.cc:237] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:17.516355 21173 catalog_manager.cc:5654] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a reported cstate change: term changed from 0 to 1, leader changed from <none> to 53b3a70556a64f3f9720cc0d6cecfb0a (127.17.169.131). New cstate: current_term: 1 leader_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } health_report { overall_health: UNKNOWN } } }
I20251213 08:59:17.533712 18086 test_util.cc:276] Using random seed: -1748178961
I20251213 08:59:17.557214 18086 test_workload.cc:405] TestWorkload: Skipping table creation because table test-workload already exists
I20251213 08:59:17.681459 21277 raft_consensus.cc:1275] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 1 FOLLOWER]: Refusing update from remote peer 53b3a70556a64f3f9720cc0d6cecfb0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:59:17.681459 21352 raft_consensus.cc:1275] T d06ff3930e7944b2bab03e288821506a P 48f1c753536c4189a1f7900702aff671 [term 1 FOLLOWER]: Refusing update from remote peer 53b3a70556a64f3f9720cc0d6cecfb0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20251213 08:59:17.684049 21479 consensus_queue.cc:1048] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251213 08:59:17.685104 21482 consensus_queue.cc:1048] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251213 08:59:17.925765 21164 maintenance_manager.cc:419] P b1219ee1e83c483795aa5c25fa3d1d14: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.003577
I20251213 08:59:18.028378 21163 maintenance_manager.cc:643] P b1219ee1e83c483795aa5c25fa3d1d14: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.102s user 0.051s sys 0.021s Metrics: {"bytes_written":6636,"cfile_init":1,"dirs.queue_time_us":23154,"dirs.run_cpu_time_us":313,"dirs.run_wall_time_us":973,"drs_written":1,"lbm_read_time_us":331,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2970,"lbm_writes_lt_1ms":27,"peak_mem_usage":0,"rows_written":5,"thread_start_us":22843,"threads_started":1}
I20251213 08:59:18.031252 21164 maintenance_manager.cc:419] P b1219ee1e83c483795aa5c25fa3d1d14: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 488 bytes on disk
I20251213 08:59:18.033344 21163 maintenance_manager.cc:643] P b1219ee1e83c483795aa5c25fa3d1d14: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":340,"lbm_reads_lt_1ms":4}
I20251213 08:59:18.371649 21452 maintenance_manager.cc:419] P 53b3a70556a64f3f9720cc0d6cecfb0a: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.242942
I20251213 08:59:18.436537 21303 maintenance_manager.cc:419] P 1646c503014749619079a5e1d3b9d033: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.242942
I20251213 08:59:18.528281 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.242942
W20251213 08:59:18.767144 21483 log.cc:927] Time spent T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20251213 08:59:19.035125 21393 maintenance_manager.cc:643] P 53b3a70556a64f3f9720cc0d6cecfb0a: FlushMRSOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.663s user 0.349s sys 0.016s Metrics: {"bytes_written":35759,"cfile_init":1,"dirs.queue_time_us":8195,"dirs.run_cpu_time_us":314,"dirs.run_wall_time_us":10633,"drs_written":1,"lbm_read_time_us":6859,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":3,"lbm_write_time_us":2551,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1200,"thread_start_us":8417,"threads_started":1}
I20251213 08:59:19.038475 21452 maintenance_manager.cc:419] P 53b3a70556a64f3f9720cc0d6cecfb0a: Scheduling UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a): 10460 bytes on disk
I20251213 08:59:19.040720 21393 maintenance_manager.cc:643] P 53b3a70556a64f3f9720cc0d6cecfb0a: UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":281,"lbm_reads_lt_1ms":4}
I20251213 08:59:19.288233 21244 maintenance_manager.cc:643] P 1646c503014749619079a5e1d3b9d033: FlushMRSOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.847s user 0.365s sys 0.029s Metrics: {"bytes_written":37112,"cfile_init":1,"dirs.queue_time_us":39114,"dirs.run_cpu_time_us":355,"dirs.run_wall_time_us":18554,"drs_written":1,"lbm_read_time_us":328,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2646,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1250,"thread_start_us":38816,"threads_started":1}
I20251213 08:59:19.291893 21303 maintenance_manager.cc:419] P 1646c503014749619079a5e1d3b9d033: Scheduling UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a): 10895 bytes on disk
I20251213 08:59:19.298384 21244 maintenance_manager.cc:643] P 1646c503014749619079a5e1d3b9d033: UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":370,"lbm_reads_lt_1ms":4}
I20251213 08:59:19.296283 21319 maintenance_manager.cc:643] P 48f1c753536c4189a1f7900702aff671: FlushMRSOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.767s user 0.407s sys 0.031s Metrics: {"bytes_written":42232,"cfile_init":1,"dirs.queue_time_us":7257,"dirs.run_cpu_time_us":321,"dirs.run_wall_time_us":7489,"drs_written":1,"lbm_read_time_us":279,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2848,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":1450,"thread_start_us":7501,"threads_started":1}
I20251213 08:59:19.308123 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a): 12614 bytes on disk
I20251213 08:59:19.324831 21319 maintenance_manager.cc:643] P 48f1c753536c4189a1f7900702aff671: UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":327,"lbm_reads_lt_1ms":4}
I20251213 08:59:20.049165 21452 maintenance_manager.cc:419] P 53b3a70556a64f3f9720cc0d6cecfb0a: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.486106
I20251213 08:59:20.308038 21303 maintenance_manager.cc:419] P 1646c503014749619079a5e1d3b9d033: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.486106
I20251213 08:59:20.332399 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.486106
W20251213 08:59:21.342391 21491 meta_cache.cc:1261] Time spent looking up entry by key: real 0.053s user 0.003s sys 0.000s
I20251213 08:59:21.372416 21393 maintenance_manager.cc:643] P 53b3a70556a64f3f9720cc0d6cecfb0a: FlushMRSOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 1.322s user 0.765s sys 0.012s Metrics: {"bytes_written":82244,"cfile_init":1,"dirs.queue_time_us":6869,"dirs.run_cpu_time_us":329,"dirs.run_wall_time_us":7319,"drs_written":1,"lbm_read_time_us":285,"lbm_reads_lt_1ms":4,"lbm_write_time_us":11159,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":2850,"thread_start_us":7087,"threads_started":1}
I20251213 08:59:21.376159 21452 maintenance_manager.cc:419] P 53b3a70556a64f3f9720cc0d6cecfb0a: Scheduling CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a): perf score=1.014383
W20251213 08:59:21.745213 21491 meta_cache.cc:1261] Time spent looking up entry by key: real 0.070s user 0.001s sys 0.000s
I20251213 08:59:21.963210 21319 maintenance_manager.cc:643] P 48f1c753536c4189a1f7900702aff671: FlushMRSOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 1.630s user 0.830s sys 0.005s Metrics: {"bytes_written":87463,"cfile_init":1,"dirs.queue_time_us":2407,"dirs.run_cpu_time_us":381,"dirs.run_wall_time_us":8526,"drs_written":1,"lbm_read_time_us":284,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3493,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3050,"thread_start_us":2675,"threads_started":1}
I20251213 08:59:21.966784 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a): 27279 bytes on disk
I20251213 08:59:21.974872 21319 maintenance_manager.cc:643] P 48f1c753536c4189a1f7900702aff671: UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":361,"lbm_reads_lt_1ms":4}
I20251213 08:59:21.977753 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a): perf score=1.014645
I20251213 08:59:22.203136 21244 maintenance_manager.cc:643] P 1646c503014749619079a5e1d3b9d033: FlushMRSOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 1.894s user 0.847s sys 0.011s Metrics: {"bytes_written":91258,"cfile_init":1,"dirs.queue_time_us":37497,"dirs.run_cpu_time_us":327,"dirs.run_wall_time_us":4515,"drs_written":1,"lbm_read_time_us":314,"lbm_reads_lt_1ms":4,"lbm_write_time_us":3669,"lbm_writes_lt_1ms":25,"peak_mem_usage":0,"rows_written":3200,"spinlock_wait_cycles":3722496,"thread_start_us":37201,"threads_started":1}
I20251213 08:59:22.207051 21303 maintenance_manager.cc:419] P 1646c503014749619079a5e1d3b9d033: Scheduling UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a): 28695 bytes on disk
I20251213 08:59:22.225749 21244 maintenance_manager.cc:643] P 1646c503014749619079a5e1d3b9d033: UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":288,"lbm_reads_lt_1ms":4}
I20251213 08:59:22.234990 21303 maintenance_manager.cc:419] P 1646c503014749619079a5e1d3b9d033: Scheduling CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a): perf score=1.014627
W20251213 08:59:22.440362 21491 meta_cache.cc:1261] Time spent looking up entry by key: real 0.086s user 0.001s sys 0.000s
I20251213 08:59:22.718392 21393 maintenance_manager.cc:643] P 53b3a70556a64f3f9720cc0d6cecfb0a: CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 1.341s user 0.797s sys 0.023s Metrics: {"bytes_written":116768,"cfile_cache_hit":12,"cfile_cache_hit_bytes":82642,"cfile_cache_miss":26,"cfile_cache_miss_bytes":198950,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":33542,"dirs.run_cpu_time_us":315,"dirs.run_wall_time_us":13221,"drs_written":1,"lbm_read_time_us":4524,"lbm_reads_lt_1ms":50,"lbm_write_time_us":3757,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":62716,"rows_written":4050,"thread_start_us":71574,"threads_started":1}
I20251213 08:59:22.721829 21452 maintenance_manager.cc:419] P 53b3a70556a64f3f9720cc0d6cecfb0a: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.486106
W20251213 08:59:23.621737 21492 meta_cache.cc:1261] Time spent looking up entry by key: real 0.056s user 0.004s sys 0.000s
I20251213 08:59:23.626785 21244 maintenance_manager.cc:643] P 1646c503014749619079a5e1d3b9d033: CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 1.391s user 0.949s sys 0.111s Metrics: {"bytes_written":127837,"cfile_cache_hit":12,"cfile_cache_hit_bytes":90270,"cfile_cache_miss":26,"cfile_cache_miss_bytes":219366,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":21037,"dirs.run_cpu_time_us":1586,"dirs.run_wall_time_us":15556,"drs_written":1,"lbm_read_time_us":4311,"lbm_reads_lt_1ms":46,"lbm_write_time_us":4027,"lbm_writes_lt_1ms":26,"num_input_rowsets":2,"peak_mem_usage":68766,"rows_written":4450,"thread_start_us":135752,"threads_started":3}
I20251213 08:59:23.629791 21303 maintenance_manager.cc:419] P 1646c503014749619079a5e1d3b9d033: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.486106
I20251213 08:59:23.710129 21319 maintenance_manager.cc:643] P 48f1c753536c4189a1f7900702aff671: CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 1.730s user 0.956s sys 0.024s Metrics: {"bytes_written":129113,"cfile_cache_hit":12,"cfile_cache_hit_bytes":92515,"cfile_cache_miss":26,"cfile_cache_miss_bytes":220377,"cfile_init":5,"delta_iterators_relevant":4,"dirs.queue_time_us":13862,"dirs.run_cpu_time_us":1575,"dirs.run_wall_time_us":10562,"drs_written":1,"lbm_read_time_us":7437,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":45,"lbm_write_time_us":4318,"lbm_writes_lt_1ms":27,"mutex_wait_us":44,"num_input_rowsets":2,"peak_mem_usage":67681,"rows_written":4500,"spinlock_wait_cycles":12819584,"thread_start_us":52285,"threads_started":3}
I20251213 08:59:23.713358 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a): 40733 bytes on disk
I20251213 08:59:23.715093 21319 maintenance_manager.cc:643] P 48f1c753536c4189a1f7900702aff671: UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":295,"lbm_reads_lt_1ms":4}
I20251213 08:59:23.716958 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.486106
I20251213 08:59:24.649865 21393 maintenance_manager.cc:643] P 53b3a70556a64f3f9720cc0d6cecfb0a: FlushMRSOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 1.927s user 0.847s sys 0.015s Metrics: {"bytes_written":91183,"cfile_init":1,"dirs.queue_time_us":24095,"dirs.run_cpu_time_us":332,"dirs.run_wall_time_us":19376,"drs_written":1,"lbm_read_time_us":289,"lbm_reads_lt_1ms":4,"lbm_write_time_us":40102,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":3200,"thread_start_us":23815,"threads_started":1}
I20251213 08:59:24.663060 21452 maintenance_manager.cc:419] P 53b3a70556a64f3f9720cc0d6cecfb0a: Scheduling CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a): perf score=1.014657
I20251213 08:59:24.872210 21244 maintenance_manager.cc:643] P 1646c503014749619079a5e1d3b9d033: FlushMRSOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 1.242s user 0.954s sys 0.012s Metrics: {"bytes_written":105207,"cfile_init":1,"dirs.queue_time_us":12425,"dirs.run_cpu_time_us":309,"dirs.run_wall_time_us":5169,"drs_written":1,"lbm_read_time_us":257,"lbm_reads_lt_1ms":4,"lbm_write_time_us":18085,"lbm_writes_1-10_ms":2,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"rows_written":3600,"thread_start_us":12146,"threads_started":1}
I20251213 08:59:24.875872 21303 maintenance_manager.cc:419] P 1646c503014749619079a5e1d3b9d033: Scheduling CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a): perf score=1.014400
I20251213 08:59:25.224427 21319 maintenance_manager.cc:643] P 48f1c753536c4189a1f7900702aff671: FlushMRSOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 1.505s user 0.991s sys 0.015s Metrics: {"bytes_written":106516,"cfile_init":1,"dirs.queue_time_us":42098,"dirs.run_cpu_time_us":260,"dirs.run_wall_time_us":3960,"drs_written":1,"lbm_read_time_us":287,"lbm_reads_lt_1ms":4,"lbm_write_time_us":15709,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":24,"peak_mem_usage":0,"rows_written":3650,"thread_start_us":41800,"threads_started":1}
I20251213 08:59:25.228214 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a): perf score=1.014400
I20251213 08:59:25.833848 18086 tablet_server.cc:178] TabletServer@127.17.169.129:0 shutting down...
W20251213 08:59:25.875741 21389 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 9 similar messages]
W20251213 08:59:25.942945 21389 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.17.169.129:36931, user_credentials={real_user=slave}} blocked reactor thread for 67217.5us
W20251213 08:59:25.945340 21389 consensus_peers.cc:597] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a -> Peer 1646c503014749619079a5e1d3b9d033 (127.17.169.129:36931): Couldn't send request to peer 1646c503014749619079a5e1d3b9d033. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20251213 08:59:26.789592 21393 maintenance_manager.cc:643] P 53b3a70556a64f3f9720cc0d6cecfb0a: CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 2.126s user 1.446s sys 0.040s Metrics: {"bytes_written":209923,"cfile_cache_hit":10,"cfile_cache_hit_bytes":132573,"cfile_cache_miss":30,"cfile_cache_miss_bytes":370889,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":21386,"dirs.run_cpu_time_us":322,"dirs.run_wall_time_us":15100,"drs_written":1,"lbm_read_time_us":12065,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":56,"lbm_write_time_us":15289,"lbm_writes_10-100_ms":1,"lbm_writes_lt_1ms":29,"num_input_rowsets":2,"peak_mem_usage":110421,"rows_written":7250,"thread_start_us":21623,"threads_started":1}
I20251213 08:59:26.794730 21452 maintenance_manager.cc:419] P 53b3a70556a64f3f9720cc0d6cecfb0a: Scheduling UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a): 67112 bytes on disk
I20251213 08:59:26.796929 21393 maintenance_manager.cc:643] P 53b3a70556a64f3f9720cc0d6cecfb0a: UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":308,"lbm_reads_lt_1ms":4}
I20251213 08:59:26.799139 21452 maintenance_manager.cc:419] P 53b3a70556a64f3f9720cc0d6cecfb0a: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.486106
I20251213 08:59:27.166301 21244 maintenance_manager.cc:643] P 1646c503014749619079a5e1d3b9d033: CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 2.290s user 1.563s sys 0.071s Metrics: {"bytes_written":231380,"cfile_cache_hit":10,"cfile_cache_hit_bytes":147529,"cfile_cache_miss":30,"cfile_cache_miss_bytes":412501,"cfile_init":7,"delta_iterators_relevant":4,"dirs.queue_time_us":63284,"dirs.run_cpu_time_us":1379,"dirs.run_wall_time_us":23106,"drs_written":1,"lbm_read_time_us":7766,"lbm_reads_1-10_ms":2,"lbm_reads_lt_1ms":56,"lbm_write_time_us":5300,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":29,"num_input_rowsets":2,"peak_mem_usage":122621,"rows_written":8050,"thread_start_us":156049,"threads_started":4}
I20251213 08:59:27.465034 21565 raft_consensus.cc:493] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 53b3a70556a64f3f9720cc0d6cecfb0a)
I20251213 08:59:27.466221 21565 raft_consensus.cc:515] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" member_type: VOTER last_known_addr { host: "127.17.169.131" port: 43595 } } peers { permanent_uuid: "1646c503014749619079a5e1d3b9d033" member_type: VOTER last_known_addr { host: "127.17.169.129" port: 36931 } } peers { permanent_uuid: "48f1c753536c4189a1f7900702aff671" member_type: VOTER last_known_addr { host: "127.17.169.130" port: 37571 } }
I20251213 08:59:27.469983 21565 leader_election.cc:290] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53b3a70556a64f3f9720cc0d6cecfb0a (127.17.169.131:43595), 48f1c753536c4189a1f7900702aff671 (127.17.169.130:37571)
W20251213 08:59:27.710503 21494 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46624: Timed out: operation timed out while waiting in prepare queue
W20251213 08:59:27.711575 21494 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46624 (ReqId={client: a904e11bc7014c8ab563455078ab3cad, seq_no=232, attempt_no=0}) took 680 ms (client timeout 499 ms). Trace:
W20251213 08:59:27.712184 21494 rpcz_store.cc:269] 1213 08:59:27.031364 (+ 0us) service_pool.cc:168] Inserting onto call queue
1213 08:59:27.031585 (+ 221us) service_pool.cc:225] Handling call
1213 08:59:27.711464 (+679879us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1213 08:59:27.710267 (+ 0us) op_driver.cc:389] HandleFailure(Timed out: operation timed out while waiting in prepare queue)
1213 08:59:27.710312 (+ 45us) write_op.cc:489] Releasing partition, row and schema locks
1213 08:59:27.710335 (+ 23us) write_op.cc:454] Released schema lock
1213 08:59:27.710355 (+ 20us) write_op.cc:335] FINISH: Op aborted
Metrics: {"child_traces":[["op",{"prepare.queue_time_us":676892}]]}
I20251213 08:59:27.846220 18086 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20251213 08:59:27.850137 18086 tablet_replica.cc:333] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033: stopping tablet replica
I20251213 08:59:27.851159 18086 raft_consensus.cc:2243] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 1 FOLLOWER]: Raft consensus shutting down.
I20251213 08:59:27.852113 18086 raft_consensus.cc:2272] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [term 1 FOLLOWER]: Raft consensus is shut down!
W20251213 08:59:27.854079 21485 meta_cache.cc:302] tablet d06ff3930e7944b2bab03e288821506a: replica 53b3a70556a64f3f9720cc0d6cecfb0a (127.17.169.131:43595) has failed: Timed out: Write RPC to 127.17.169.131:43595 timed out after 0.500s (SENT)
W20251213 08:59:27.854815 21485 batcher.cc:441] Timed out: Failed to write batch of 50 ops to tablet d06ff3930e7944b2bab03e288821506a after 1 attempt(s): Failed to write to server: 53b3a70556a64f3f9720cc0d6cecfb0a (127.17.169.131:43595): Write RPC to 127.17.169.131:43595 timed out after 0.500s (SENT)
W20251213 08:59:27.868440 21568 rpcz_store.cc:267] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:46624 (ReqId={client: a904e11bc7014c8ab563455078ab3cad, seq_no=231, attempt_no=0}) took 839 ms (client timeout 499 ms). Trace:
W20251213 08:59:27.869081 21568 rpcz_store.cc:269] 1213 08:59:27.028616 (+ 0us) service_pool.cc:168] Inserting onto call queue
1213 08:59:27.028775 (+ 159us) service_pool.cc:225] Handling call
1213 08:59:27.868251 (+839476us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1213 08:59:27.052768 (+ 0us) write_op.cc:183] PREPARE: starting on tablet d06ff3930e7944b2bab03e288821506a
1213 08:59:27.053257 (+ 489us) write_op.cc:432] Acquiring schema lock in shared mode
1213 08:59:27.053293 (+ 36us) write_op.cc:435] Acquired schema lock
1213 08:59:27.053307 (+ 14us) tablet.cc:660] Decoding operations
1213 08:59:27.062099 (+ 8792us) write_op.cc:620] Acquiring the partition lock for write op
1213 08:59:27.062150 (+ 51us) write_op.cc:641] Partition lock acquired for write op
1213 08:59:27.062196 (+ 46us) tablet.cc:683] Acquiring locks for 50 operations
1213 08:59:27.064038 (+ 1842us) tablet.cc:699] Row locks acquired
1213 08:59:27.064056 (+ 18us) write_op.cc:260] PREPARE: finished
1213 08:59:27.064250 (+ 194us) write_op.cc:270] Start()
1213 08:59:27.064354 (+ 104us) write_op.cc:276] Timestamp: P: 1765616367064208 usec, L: 0
1213 08:59:27.064371 (+ 17us) op_driver.cc:348] REPLICATION: starting
1213 08:59:27.078935 (+ 14564us) log.cc:844] Serialized 2467 byte log entry
1213 08:59:27.709914 (+630979us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b54002067d4. stack: 00007fb819740083 00007fb81973ff80 00007fb819092574 0000000000351244 00000000003511d9 00007fb8200da5a1 00007fb82016caa8 00007fb8205de4d1 00007fb8205ddca8 00007fb8205e2941 00007fb8205e28f9 00007fb8205e2889 00007fb8205e2851 00007fb8205e1a7d 00000000003664f4 0000000000366339
1213 08:59:27.836693 (+126779us) op_driver.cc:464] REPLICATION: finished
1213 08:59:27.852753 (+ 16060us) write_op.cc:301] APPLY: starting
1213 08:59:27.853572 (+ 819us) tablet.cc:1365] starting BulkCheckPresence
1213 08:59:27.859118 (+ 5546us) tablet.cc:1368] finished BulkCheckPresence
1213 08:59:27.859148 (+ 30us) tablet.cc:1370] starting ApplyRowOperation cycle
1213 08:59:27.866176 (+ 7028us) tablet.cc:1381] finished ApplyRowOperation cycle
1213 08:59:27.866659 (+ 483us) tablet_metrics.cc:563] ProbeStats: bloom_lookups=50,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=50
1213 08:59:27.866706 (+ 47us) write_op.cc:312] APPLY: finished
1213 08:59:27.867355 (+ 649us) log.cc:844] Serialized 320 byte log entry
1213 08:59:27.867549 (+ 194us) write_op.cc:489] Releasing partition, row and schema locks
1213 08:59:27.867826 (+ 277us) write_op.cc:454] Released schema lock
1213 08:59:27.867917 (+ 91us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"child_traces":[["op",{"apply.queue_time_us":15814,"cfile_cache_hit":6,"cfile_cache_hit_bytes":20596,"num_ops":50,"prepare.queue_time_us":22346,"prepare.run_cpu_time_us":32880,"prepare.run_wall_time_us":657302,"replication_time_us":772180,"spinlock_wait_cycles":31900928,"thread_start_us":16043,"threads_started":1}]]}
I20251213 08:59:27.945860 21352 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06ff3930e7944b2bab03e288821506a" candidate_uuid: "1646c503014749619079a5e1d3b9d033" candidate_term: 2 candidate_status { last_received { term: 1 index: 202 } } ignore_live_leader: false dest_uuid: "48f1c753536c4189a1f7900702aff671" is_pre_election: true
W20251213 08:59:27.965104 21572 leader_election.cc:336] T d06ff3930e7944b2bab03e288821506a P 1646c503014749619079a5e1d3b9d033 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 48f1c753536c4189a1f7900702aff671 (127.17.169.130:37571): Service unavailable: reactor is shutting down (error 108)
W20251213 08:59:27.966491 21317 connection.cc:537] server connection from 127.0.0.1:47938 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
F20251213 08:59:27.990365 18086 leader_election.cc:206] Check failed: has_responded_
I20251213 08:59:28.011171 21426 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d06ff3930e7944b2bab03e288821506a" candidate_uuid: "1646c503014749619079a5e1d3b9d033" candidate_term: 2 candidate_status { last_received { term: 1 index: 202 } } ignore_live_leader: false dest_uuid: "53b3a70556a64f3f9720cc0d6cecfb0a" is_pre_election: true
*** Check failure stack trace: ***
*** Aborted at 1765616368 (unix time) try "date -d @1765616368" if you are using GNU date ***
W20251213 08:59:28.055428 21390 connection.cc:441] server connection from 127.0.0.1:40930 torn down before Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40930 (request call id 0) could send its response
PC: @ 0x0 (unknown)
*** SIGABRT (@0x3e8000046a6) received by PID 18086 (TID 0x7fb8150a4840) from PID 18086; stack trace: ***
@ 0x2c5d90 __tsan::CallUserSignalHandler() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:1915
@ 0x2c8273 rtl_sigaction() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:2005
@ 0x7fb81abb5980 (unknown) at ??:0
W20251213 08:59:28.113705 21298 debug-util.cc:398] Leaking SignalData structure 0x7b080019d340 after lost signal to thread 18089
W20251213 08:59:28.114779 21298 debug-util.cc:398] Leaking SignalData structure 0x7b0800130c20 after lost signal to thread 21209
@ 0x7fb815d62fb7 gsignal at ??:0
@ 0x7fb815d64921 abort at ??:0
@ 0x2cbd86 abort at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/src/llvm-11.0.0.src/projects/compiler-rt/lib/tsan/rtl/tsan_interceptors_posix.cpp:263
@ 0x7fb818dfada4 google::LogMessage::SendToLog() at ??:0
@ 0x7fb818dfb950 google::LogMessage::Flush() at ??:0
@ 0x7fb818e00a8b google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fb82011d9cc kudu::consensus::LeaderElection::~LeaderElection() at ??:0
@ 0x7fb82012b20f kudu::RefCountedThreadSafe<>::DeleteInternal() at ??:0
@ 0x7fb82012b1da kudu::DefaultRefCountedThreadSafeTraits<>::Destruct() at ??:0
W20251213 08:59:28.623157 21389 consensus_peers.cc:597] T d06ff3930e7944b2bab03e288821506a P 53b3a70556a64f3f9720cc0d6cecfb0a -> Peer 1646c503014749619079a5e1d3b9d033 (127.17.169.129:36931): Couldn't send request to peer 1646c503014749619079a5e1d3b9d033. Status: Network error: Client connection negotiation failed: client connection to 127.17.169.129:36931: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
@ 0x7fb82012b196 kudu::RefCountedThreadSafe<>::Release() at ??:0
@ 0x7fb820122c67 scoped_refptr<>::~scoped_refptr() at ??:0
I20251213 08:59:28.823496 21319 maintenance_manager.cc:643] P 48f1c753536c4189a1f7900702aff671: CompactRowSetsOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 3.591s user 1.618s sys 0.074s Metrics: {"bytes_written":233654,"cfile_cache_hit":10,"cfile_cache_hit_bytes":149426,"cfile_cache_miss":32,"cfile_cache_miss_bytes":417896,"cfile_init":6,"delta_iterators_relevant":4,"dirs.queue_time_us":47771,"dirs.run_cpu_time_us":375,"dirs.run_wall_time_us":1817,"drs_written":1,"lbm_read_time_us":42553,"lbm_reads_1-10_ms":3,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":52,"lbm_write_time_us":5372,"lbm_writes_lt_1ms":30,"num_input_rowsets":2,"peak_mem_usage":124144,"rows_written":8150,"thread_start_us":46705,"threads_started":1}
I20251213 08:59:28.827821 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a): 75490 bytes on disk
@ 0x7fb82011f003 kudu::consensus::LeaderElection::Run()::$_0::~Run() at ??:0
I20251213 08:59:28.830365 21319 maintenance_manager.cc:643] P 48f1c753536c4189a1f7900702aff671: UndoDeltaBlockGCOp(d06ff3930e7944b2bab03e288821506a) complete. Timing: real 0.001s user 0.001s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":299,"lbm_reads_lt_1ms":4}
I20251213 08:59:28.832415 21378 maintenance_manager.cc:419] P 48f1c753536c4189a1f7900702aff671: Scheduling FlushMRSOp(d06ff3930e7944b2bab03e288821506a): perf score=0.486106
@ 0x7fb820120bea std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
@ 0x7fb82012150a std::__1::__function::__alloc_func<>::destroy() at ??:0
@ 0x7fb8201207db std::__1::__function::__func<>::destroy_deallocate() at ??:0
@ 0x35cbbb std::__1::__function::__value_func<>::~__value_func() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:?
@ 0x34a7fa std::__1::function<>::~function() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2543
@ 0x7fb81e2e487e kudu::rpc::Proxy::RefreshDnsAndEnqueueRequest()::$_0::~RefreshDnsAndEnqueueRequest() at ??:0
@ 0x7fb81e2e5c3a std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
@ 0x7fb81e2e656a std::__1::__function::__alloc_func<>::destroy() at ??:0
@ 0x7fb81e2e580b std::__1::__function::__func<>::destroy_deallocate() at ??:0
@ 0x7fb8225eceab std::__1::__function::__value_func<>::~__value_func() at ??:0
@ 0x7fb8225bd91a std::__1::function<>::~function() at ??:0
@ 0x7fb8196e869b kudu::DnsResolver::RefreshAddressesAsync()::$_1::~RefreshAddressesAsync() at ??:0
@ 0x7fb8196ea89a std::__1::__compressed_pair_elem<>::~__compressed_pair_elem() at ??:0
@ 0x7fb8196eb20a std::__1::__function::__alloc_func<>::destroy() at ??:0
@ 0x7fb8196ea43b std::__1::__function::__func<>::destroy_deallocate() at ??:0
@ 0x35cbbb std::__1::__function::__value_func<>::~__value_func() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:?
@ 0x34a7fa std::__1::function<>::~function() at /home/jenkins-slave/workspace/build_and_test_flaky@2/thirdparty/installed/tsan/include/c++/v1/functional:2543