Note: This is test shard 1 of 8.
[==========] Running 2 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 1 test from MaintenanceModeRF3ITest
[ RUN ] MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate
2025-12-11T20:54:49Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-12-11T20:54:49Z Disabled control of system clock
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20251211 20:54:49.487445 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.211.254:40613
--webserver_interface=127.5.211.254
--webserver_port=0
--builtin_ntp_servers=127.5.211.212:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.211.254:40613 with env {}
W20251211 20:54:49.792826 5975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:54:49.793113 5975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:54:49.793233 5975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:54:49.800943 5975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251211 20:54:49.801023 5975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:54:49.801095 5975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251211 20:54:49.801174 5975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251211 20:54:49.810083 5975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.211.254:40613
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.211.254:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.211.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:54:49.811537 5975 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:54:49.813606 5975 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:54:49.821550 5981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:54:49.821583 5980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:54:49.822453 5983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:54:49.822757 5975 server_base.cc:1047] running on GCE node
I20251211 20:54:49.823513 5975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:54:49.824695 5975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:54:49.825896 5975 hybrid_clock.cc:648] HybridClock initialized: now 1765486489825827 us; error 59 us; skew 500 ppm
I20251211 20:54:49.826243 5975 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:54:49.828378 5975 webserver.cc:492] Webserver started at http://127.5.211.254:37167/ using document root <none> and password file <none>
I20251211 20:54:49.829021 5975 fs_manager.cc:362] Metadata directory not provided
I20251211 20:54:49.829106 5975 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:54:49.829420 5975 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:54:49.831593 5975 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data/instance:
uuid: "e4d4d4e72e654afe9bf24bdad38cef52"
format_stamp: "Formatted at 2025-12-11 20:54:49 on dist-test-slave-sk6m"
I20251211 20:54:49.832293 5975 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal/instance:
uuid: "e4d4d4e72e654afe9bf24bdad38cef52"
format_stamp: "Formatted at 2025-12-11 20:54:49 on dist-test-slave-sk6m"
I20251211 20:54:49.837011 5975 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20251211 20:54:49.840077 5989 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:49.841398 5975 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20251211 20:54:49.841563 5975 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
uuid: "e4d4d4e72e654afe9bf24bdad38cef52"
format_stamp: "Formatted at 2025-12-11 20:54:49 on dist-test-slave-sk6m"
I20251211 20:54:49.841737 5975 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:54:49.862725 5975 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:54:49.863512 5975 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:54:49.863780 5975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:54:49.882969 5975 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.254:40613
I20251211 20:54:49.883002 6040 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.254:40613 every 8 connection(s)
I20251211 20:54:49.884290 5975 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data/info.pb
I20251211 20:54:49.888245 6041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:49.890758 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 5975
I20251211 20:54:49.891044 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal/instance
I20251211 20:54:49.899186 6041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: Bootstrap starting.
I20251211 20:54:49.901970 6041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:49.902874 6041 log.cc:826] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: Log is configured to *not* fsync() on all Append() calls
I20251211 20:54:49.905491 6041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: No bootstrap required, opened a new log
I20251211 20:54:49.911441 6041 raft_consensus.cc:359] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } }
I20251211 20:54:49.911851 6041 raft_consensus.cc:385] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:49.911962 6041 raft_consensus.cc:740] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4d4d4e72e654afe9bf24bdad38cef52, State: Initialized, Role: FOLLOWER
I20251211 20:54:49.912554 6041 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } }
I20251211 20:54:49.912750 6041 raft_consensus.cc:399] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251211 20:54:49.912875 6041 raft_consensus.cc:493] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251211 20:54:49.912992 6041 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:49.915856 6041 raft_consensus.cc:515] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } }
I20251211 20:54:49.916373 6041 leader_election.cc:304] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4d4d4e72e654afe9bf24bdad38cef52; no voters:
I20251211 20:54:49.917029 6041 leader_election.cc:290] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251211 20:54:49.917105 6046 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:54:49.918386 6046 raft_consensus.cc:697] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 1 LEADER]: Becoming Leader. State: Replica: e4d4d4e72e654afe9bf24bdad38cef52, State: Running, Role: LEADER
I20251211 20:54:49.918884 6041 sys_catalog.cc:565] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: configured and running, proceeding with master startup.
I20251211 20:54:49.918853 6046 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } }
I20251211 20:54:49.921504 6048 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4d4d4e72e654afe9bf24bdad38cef52. Latest consensus state: current_term: 1 leader_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } } }
I20251211 20:54:49.921777 6048 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: This master's current role is: LEADER
I20251211 20:54:49.922291 6047 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } } }
I20251211 20:54:49.922559 6047 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: This master's current role is: LEADER
I20251211 20:54:49.928521 6055 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251211 20:54:49.933341 6055 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251211 20:54:49.940795 6055 catalog_manager.cc:1357] Generated new cluster ID: 533ab04ce03b4bf4b47ded938756be0f
I20251211 20:54:49.940898 6055 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251211 20:54:49.954660 6055 catalog_manager.cc:1380] Generated new certificate authority record
I20251211 20:54:49.956054 6055 catalog_manager.cc:1514] Loading token signing keys...
I20251211 20:54:49.970815 6055 catalog_manager.cc:6027] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: Generated new TSK 0
I20251211 20:54:49.971772 6055 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251211 20:54:49.978806 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.193:0
--local_ip_for_outbound_sockets=127.5.211.193
--webserver_interface=127.5.211.193
--webserver_port=0
--tserver_master_addrs=127.5.211.254:40613
--builtin_ntp_servers=127.5.211.212:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--follower_unavailable_considered_failed_sec=2
--enable_log_gc=false with env {}
W20251211 20:54:50.277405 6066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:54:50.277676 6066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:54:50.277781 6066 flags.cc:425] Enabled unsafe flag: --enable_log_gc=false
W20251211 20:54:50.277864 6066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:54:50.285400 6066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:54:50.285580 6066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.193
I20251211 20:54:50.294827 6066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=2
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.211.193
--webserver_port=0
--enable_log_gc=false
--tserver_master_addrs=127.5.211.254:40613
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.193
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:54:50.296244 6066 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:54:50.297851 6066 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:54:50.306420 6072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:54:50.306456 6071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:54:50.307242 6074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:54:50.307730 6066 server_base.cc:1047] running on GCE node
I20251211 20:54:50.308262 6066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:54:50.309021 6066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:54:50.310235 6066 hybrid_clock.cc:648] HybridClock initialized: now 1765486490310162 us; error 46 us; skew 500 ppm
I20251211 20:54:50.310520 6066 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:54:50.312443 6066 webserver.cc:492] Webserver started at http://127.5.211.193:37369/ using document root <none> and password file <none>
I20251211 20:54:50.313077 6066 fs_manager.cc:362] Metadata directory not provided
I20251211 20:54:50.313205 6066 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:54:50.313505 6066 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:54:50.315541 6066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data/instance:
uuid: "cd482952c9fd4a1fb75c31f128338e89"
format_stamp: "Formatted at 2025-12-11 20:54:50 on dist-test-slave-sk6m"
I20251211 20:54:50.316180 6066 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal/instance:
uuid: "cd482952c9fd4a1fb75c31f128338e89"
format_stamp: "Formatted at 2025-12-11 20:54:50 on dist-test-slave-sk6m"
I20251211 20:54:50.320887 6066 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20251211 20:54:50.324010 6080 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:50.325196 6066 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251211 20:54:50.325354 6066 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
uuid: "cd482952c9fd4a1fb75c31f128338e89"
format_stamp: "Formatted at 2025-12-11 20:54:50 on dist-test-slave-sk6m"
I20251211 20:54:50.325515 6066 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:54:50.341845 6066 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:54:50.342573 6066 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:54:50.342830 6066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:54:50.343586 6066 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:54:50.344933 6066 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:54:50.345016 6066 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:50.345126 6066 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:54:50.345180 6066 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:50.375224 6066 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.193:34639
I20251211 20:54:50.375259 6192 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.193:34639 every 8 connection(s)
I20251211 20:54:50.376451 6066 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
I20251211 20:54:50.382784 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 6066
I20251211 20:54:50.382994 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal/instance
I20251211 20:54:50.386190 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.194:0
--local_ip_for_outbound_sockets=127.5.211.194
--webserver_interface=127.5.211.194
--webserver_port=0
--tserver_master_addrs=127.5.211.254:40613
--builtin_ntp_servers=127.5.211.212:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--follower_unavailable_considered_failed_sec=2
--enable_log_gc=false with env {}
I20251211 20:54:50.388379 6193 heartbeater.cc:344] Connected to a master server at 127.5.211.254:40613
I20251211 20:54:50.388711 6193 heartbeater.cc:461] Registering TS with master...
I20251211 20:54:50.389663 6193 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:54:50.391798 6006 ts_manager.cc:194] Registered new tserver with Master: cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639)
I20251211 20:54:50.393203 6006 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.193:44285
W20251211 20:54:50.689184 6196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:54:50.689486 6196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:54:50.689597 6196 flags.cc:425] Enabled unsafe flag: --enable_log_gc=false
W20251211 20:54:50.689692 6196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:54:50.697438 6196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:54:50.697613 6196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.194
I20251211 20:54:50.706568 6196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=2
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.211.194
--webserver_port=0
--enable_log_gc=false
--tserver_master_addrs=127.5.211.254:40613
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.194
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:54:50.708006 6196 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:54:50.709627 6196 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:54:50.718369 6202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:54:50.718335 6201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:54:50.719197 6204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:54:50.719637 6196 server_base.cc:1047] running on GCE node
I20251211 20:54:50.720206 6196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:54:50.721036 6196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:54:50.722265 6196 hybrid_clock.cc:648] HybridClock initialized: now 1765486490722208 us; error 34 us; skew 500 ppm
I20251211 20:54:50.722563 6196 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:54:50.724545 6196 webserver.cc:492] Webserver started at http://127.5.211.194:41825/ using document root <none> and password file <none>
I20251211 20:54:50.725221 6196 fs_manager.cc:362] Metadata directory not provided
I20251211 20:54:50.725325 6196 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:54:50.725628 6196 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:54:50.727823 6196 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/data/instance:
uuid: "4acae5cddb8e4a9b8777025ea93a4b77"
format_stamp: "Formatted at 2025-12-11 20:54:50 on dist-test-slave-sk6m"
I20251211 20:54:50.728518 6196 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/wal/instance:
uuid: "4acae5cddb8e4a9b8777025ea93a4b77"
format_stamp: "Formatted at 2025-12-11 20:54:50 on dist-test-slave-sk6m"
I20251211 20:54:50.733470 6196 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20251211 20:54:50.736572 6210 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:50.737767 6196 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251211 20:54:50.737916 6196 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/wal
uuid: "4acae5cddb8e4a9b8777025ea93a4b77"
format_stamp: "Formatted at 2025-12-11 20:54:50 on dist-test-slave-sk6m"
I20251211 20:54:50.738143 6196 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:54:50.781440 6196 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:54:50.782320 6196 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:54:50.782575 6196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:54:50.783408 6196 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:54:50.784888 6196 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:54:50.784972 6196 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:50.785096 6196 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:54:50.785151 6196 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:50.817188 6196 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.194:45317
I20251211 20:54:50.817219 6322 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.194:45317 every 8 connection(s)
I20251211 20:54:50.818598 6196 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
I20251211 20:54:50.821594 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 6196
I20251211 20:54:50.821727 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-1/wal/instance
I20251211 20:54:50.825381 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.195:0
--local_ip_for_outbound_sockets=127.5.211.195
--webserver_interface=127.5.211.195
--webserver_port=0
--tserver_master_addrs=127.5.211.254:40613
--builtin_ntp_servers=127.5.211.212:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--follower_unavailable_considered_failed_sec=2
--enable_log_gc=false with env {}
I20251211 20:54:50.832401 6323 heartbeater.cc:344] Connected to a master server at 127.5.211.254:40613
I20251211 20:54:50.832789 6323 heartbeater.cc:461] Registering TS with master...
I20251211 20:54:50.833529 6323 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:54:50.834885 6006 ts_manager.cc:194] Registered new tserver with Master: 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:54:50.835695 6006 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.194:57523
W20251211 20:54:51.126096 6326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:54:51.126367 6326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:54:51.126471 6326 flags.cc:425] Enabled unsafe flag: --enable_log_gc=false
W20251211 20:54:51.126565 6326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:54:51.134297 6326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:54:51.134500 6326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.195
I20251211 20:54:51.143760 6326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=2
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.211.195
--webserver_port=0
--enable_log_gc=false
--tserver_master_addrs=127.5.211.254:40613
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.195
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:54:51.145330 6326 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:54:51.147167 6326 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:54:51.156478 6331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:54:51.156404 6334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:54:51.156514 6332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:54:51.157326 6326 server_base.cc:1047] running on GCE node
I20251211 20:54:51.157940 6326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:54:51.158834 6326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:54:51.160035 6326 hybrid_clock.cc:648] HybridClock initialized: now 1765486491159981 us; error 33 us; skew 500 ppm
I20251211 20:54:51.160326 6326 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:54:51.162585 6326 webserver.cc:492] Webserver started at http://127.5.211.195:46155/ using document root <none> and password file <none>
I20251211 20:54:51.163285 6326 fs_manager.cc:362] Metadata directory not provided
I20251211 20:54:51.163403 6326 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:54:51.163723 6326 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:54:51.165982 6326 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/data/instance:
uuid: "d60a4040120049e48295adbd7c33e3e4"
format_stamp: "Formatted at 2025-12-11 20:54:51 on dist-test-slave-sk6m"
I20251211 20:54:51.166693 6326 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/wal/instance:
uuid: "d60a4040120049e48295adbd7c33e3e4"
format_stamp: "Formatted at 2025-12-11 20:54:51 on dist-test-slave-sk6m"
I20251211 20:54:51.171538 6326 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20251211 20:54:51.174620 6340 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:51.175745 6326 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251211 20:54:51.175889 6326 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/wal
uuid: "d60a4040120049e48295adbd7c33e3e4"
format_stamp: "Formatted at 2025-12-11 20:54:51 on dist-test-slave-sk6m"
I20251211 20:54:51.176074 6326 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:54:51.191921 6326 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:54:51.192602 6326 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:54:51.192869 6326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:54:51.193645 6326 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:54:51.195070 6326 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:54:51.195147 6326 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:51.195264 6326 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:54:51.195319 6326 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:51.228657 6326 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.195:46117
I20251211 20:54:51.228685 6452 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.195:46117 every 8 connection(s)
I20251211 20:54:51.230224 6326 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
I20251211 20:54:51.231348 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 6326
I20251211 20:54:51.231534 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-2/wal/instance
I20251211 20:54:51.246825 6453 heartbeater.cc:344] Connected to a master server at 127.5.211.254:40613
I20251211 20:54:51.247171 6453 heartbeater.cc:461] Registering TS with master...
I20251211 20:54:51.247928 6453 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:54:51.249258 6006 ts_manager.cc:194] Registered new tserver with Master: d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.250020 6006 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.195:60903
I20251211 20:54:51.260172 5967 external_mini_cluster.cc:949] 3 TS(s) registered with all masters
I20251211 20:54:51.276888 5967 test_util.cc:276] Using random seed: 1529550389
I20251211 20:54:51.296619 6006 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39450:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\004\001\000UUU\025\004\001\000\252\252\252*\004\001\000\377\377\377?\004\001\000TUUU\004\001\000\251\252\252j"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20251211 20:54:51.298303 6006 catalog_manager.cc:7016] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20251211 20:54:51.332429 6386 tablet_service.cc:1505] Processing CreateTablet for tablet a9dbb37290964ca7ba650d2de1404245 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251211 20:54:51.333218 6125 tablet_service.cc:1505] Processing CreateTablet for tablet 4927adf430d04b6199dbbfaf1de1c16d (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251211 20:54:51.333825 6387 tablet_service.cc:1505] Processing CreateTablet for tablet d915f836ed2c479c9d6623e11266ed55 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251211 20:54:51.334556 6387 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d915f836ed2c479c9d6623e11266ed55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.333178 6128 tablet_service.cc:1505] Processing CreateTablet for tablet e7ef1dab1f4c47379f2756f162302ecc (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251211 20:54:51.334659 6125 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4927adf430d04b6199dbbfaf1de1c16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.333581 6126 tablet_service.cc:1505] Processing CreateTablet for tablet a9dbb37290964ca7ba650d2de1404245 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251211 20:54:51.334429 6253 tablet_service.cc:1505] Processing CreateTablet for tablet ce1e22ac4b814a508b20688b15d0c959 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251211 20:54:51.335234 6126 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9dbb37290964ca7ba650d2de1404245. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.335680 6253 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1e22ac4b814a508b20688b15d0c959. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.335733 6128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7ef1dab1f4c47379f2756f162302ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.334030 6258 tablet_service.cc:1505] Processing CreateTablet for tablet e7ef1dab1f4c47379f2756f162302ecc (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251211 20:54:51.336768 6258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7ef1dab1f4c47379f2756f162302ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.333034 6127 tablet_service.cc:1505] Processing CreateTablet for tablet d915f836ed2c479c9d6623e11266ed55 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251211 20:54:51.337386 6127 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d915f836ed2c479c9d6623e11266ed55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.337414 6254 tablet_service.cc:1505] Processing CreateTablet for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251211 20:54:51.337431 6257 tablet_service.cc:1505] Processing CreateTablet for tablet d915f836ed2c479c9d6623e11266ed55 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 357913941 <= VALUES < 715827882
I20251211 20:54:51.337966 6257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d915f836ed2c479c9d6623e11266ed55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.338207 6388 tablet_service.cc:1505] Processing CreateTablet for tablet e7ef1dab1f4c47379f2756f162302ecc (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION VALUES < 357913941
I20251211 20:54:51.338400 6384 tablet_service.cc:1505] Processing CreateTablet for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251211 20:54:51.338543 6254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.338704 6388 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7ef1dab1f4c47379f2756f162302ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.338968 6384 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.339414 6255 tablet_service.cc:1505] Processing CreateTablet for tablet 4927adf430d04b6199dbbfaf1de1c16d (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251211 20:54:51.339803 6255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4927adf430d04b6199dbbfaf1de1c16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.340471 6256 tablet_service.cc:1505] Processing CreateTablet for tablet a9dbb37290964ca7ba650d2de1404245 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 1073741823
I20251211 20:54:51.340922 6256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9dbb37290964ca7ba650d2de1404245. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.332562 6385 tablet_service.cc:1505] Processing CreateTablet for tablet 4927adf430d04b6199dbbfaf1de1c16d (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 1073741823 <= VALUES < 1431655764
I20251211 20:54:51.343097 6385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4927adf430d04b6199dbbfaf1de1c16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.346729 6383 tablet_service.cc:1505] Processing CreateTablet for tablet ce1e22ac4b814a508b20688b15d0c959 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251211 20:54:51.347352 6383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1e22ac4b814a508b20688b15d0c959. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.352305 6124 tablet_service.cc:1505] Processing CreateTablet for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1789569705
I20251211 20:54:51.352783 6124 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.353979 6477 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4: Bootstrap starting.
I20251211 20:54:51.354518 6123 tablet_service.cc:1505] Processing CreateTablet for tablet ce1e22ac4b814a508b20688b15d0c959 (DEFAULT_TABLE table=test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]), partition=RANGE (key) PARTITION 1789569705 <= VALUES
I20251211 20:54:51.354972 6123 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1e22ac4b814a508b20688b15d0c959. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.355381 6476 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:54:51.359437 6476 tablet_bootstrap.cc:654] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.360884 6477 tablet_bootstrap.cc:654] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.360841 6476 log.cc:826] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Log is configured to *not* fsync() on all Append() calls
I20251211 20:54:51.361133 6386 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9dbb37290964ca7ba650d2de1404245. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:54:51.362371 6477 log.cc:826] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4: Log is configured to *not* fsync() on all Append() calls
I20251211 20:54:51.395637 6193 heartbeater.cc:499] Master 127.5.211.254:40613 was elected leader, sending a full tablet report...
I20251211 20:54:51.407938 6480 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77: Bootstrap starting.
I20251211 20:54:51.410702 6480 tablet_bootstrap.cc:654] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.411844 6480 log.cc:826] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77: Log is configured to *not* fsync() on all Append() calls
I20251211 20:54:51.412875 6476 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: No bootstrap required, opened a new log
I20251211 20:54:51.413122 6476 ts_tablet_manager.cc:1403] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 0.058s user 0.016s sys 0.033s
I20251211 20:54:51.414439 6477 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4: No bootstrap required, opened a new log
I20251211 20:54:51.414752 6477 ts_tablet_manager.cc:1403] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4: Time spent bootstrapping tablet: real 0.061s user 0.030s sys 0.024s
I20251211 20:54:51.416363 6480 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77: No bootstrap required, opened a new log
I20251211 20:54:51.416584 6480 ts_tablet_manager.cc:1403] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20251211 20:54:51.417935 6476 raft_consensus.cc:359] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.418267 6476 raft_consensus.cc:385] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.418373 6476 raft_consensus.cc:740] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.418929 6476 consensus_queue.cc:260] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.420481 6476 ts_tablet_manager.cc:1434] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20251211 20:54:51.420841 6476 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:54:51.421298 6477 raft_consensus.cc:359] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.421630 6477 raft_consensus.cc:385] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.421729 6477 raft_consensus.cc:740] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60a4040120049e48295adbd7c33e3e4, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.422703 6477 consensus_queue.cc:260] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.423008 6480 raft_consensus.cc:359] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.423382 6480 raft_consensus.cc:385] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.423491 6480 raft_consensus.cc:740] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acae5cddb8e4a9b8777025ea93a4b77, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.423906 6476 tablet_bootstrap.cc:654] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.424194 6480 consensus_queue.cc:260] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.424928 6453 heartbeater.cc:499] Master 127.5.211.254:40613 was elected leader, sending a full tablet report...
I20251211 20:54:51.425429 6477 ts_tablet_manager.cc:1434] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20251211 20:54:51.425750 6323 heartbeater.cc:499] Master 127.5.211.254:40613 was elected leader, sending a full tablet report...
I20251211 20:54:51.425853 6480 ts_tablet_manager.cc:1434] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent starting tablet: real 0.009s user 0.009s sys 0.001s
I20251211 20:54:51.426051 6477 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4: Bootstrap starting.
I20251211 20:54:51.426286 6480 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77: Bootstrap starting.
I20251211 20:54:51.428189 6480 tablet_bootstrap.cc:654] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.428567 6476 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: No bootstrap required, opened a new log
I20251211 20:54:51.428690 6476 ts_tablet_manager.cc:1403] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.002s
I20251211 20:54:51.429697 6477 tablet_bootstrap.cc:654] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.429766 6476 raft_consensus.cc:359] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.430018 6476 raft_consensus.cc:385] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.430227 6476 raft_consensus.cc:740] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.430594 6476 consensus_queue.cc:260] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.431216 6476 ts_tablet_manager.cc:1434] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:54:51.431571 6476 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:54:51.433736 6476 tablet_bootstrap.cc:654] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.434801 6477 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4: No bootstrap required, opened a new log
I20251211 20:54:51.434947 6477 ts_tablet_manager.cc:1403] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20251211 20:54:51.435842 6477 raft_consensus.cc:359] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.436004 6477 raft_consensus.cc:385] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.436105 6477 raft_consensus.cc:740] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60a4040120049e48295adbd7c33e3e4, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.436358 6477 consensus_queue.cc:260] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.436765 6482 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.436987 6482 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.438920 6477 ts_tablet_manager.cc:1434] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s
I20251211 20:54:51.439422 6482 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.441018 6477 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4: Bootstrap starting.
I20251211 20:54:51.442889 6477 tablet_bootstrap.cc:654] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.445317 6480 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77: No bootstrap required, opened a new log
I20251211 20:54:51.445482 6480 ts_tablet_manager.cc:1403] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.012s
I20251211 20:54:51.446687 6480 raft_consensus.cc:359] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.446976 6480 raft_consensus.cc:385] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.447100 6477 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4: No bootstrap required, opened a new log
I20251211 20:54:51.447099 6480 raft_consensus.cc:740] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acae5cddb8e4a9b8777025ea93a4b77, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.447288 6477 ts_tablet_manager.cc:1403] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20251211 20:54:51.447428 6480 consensus_queue.cc:260] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.448060 6480 ts_tablet_manager.cc:1434] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251211 20:54:51.448295 6477 raft_consensus.cc:359] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.448444 6480 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77: Bootstrap starting.
I20251211 20:54:51.448491 6477 raft_consensus.cc:385] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.448563 6477 raft_consensus.cc:740] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60a4040120049e48295adbd7c33e3e4, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.448833 6477 consensus_queue.cc:260] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.450702 6480 tablet_bootstrap.cc:654] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.451145 6483 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.451370 6483 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.453459 6483 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:54:51.457450 6483 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.457595 6483 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.458096 6483 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:54:51.460194 6477 ts_tablet_manager.cc:1434] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4: Time spent starting tablet: real 0.013s user 0.001s sys 0.001s
I20251211 20:54:51.460570 6477 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4: Bootstrap starting.
I20251211 20:54:51.462338 6477 tablet_bootstrap.cc:654] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.468464 6476 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: No bootstrap required, opened a new log
I20251211 20:54:51.468683 6476 ts_tablet_manager.cc:1403] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 0.037s user 0.018s sys 0.017s
I20251211 20:54:51.469682 6476 raft_consensus.cc:359] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.469923 6476 raft_consensus.cc:385] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.470052 6476 raft_consensus.cc:740] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.470451 6476 consensus_queue.cc:260] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.471091 6476 ts_tablet_manager.cc:1434] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20251211 20:54:51.471490 6476 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:54:51.473472 6476 tablet_bootstrap.cc:654] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Neither blocks nor log segments found. Creating new log.
W20251211 20:54:51.482074 6454 tablet.cc:2378] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:54:51.482228 6148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89" is_pre_election: true
I20251211 20:54:51.482652 6148 raft_consensus.cc:2468] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60a4040120049e48295adbd7c33e3e4 in term 0.
I20251211 20:54:51.483242 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89" is_pre_election: true
I20251211 20:54:51.483819 6477 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4: No bootstrap required, opened a new log
I20251211 20:54:51.483970 6477 ts_tablet_manager.cc:1403] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.000s
I20251211 20:54:51.484591 6344 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89, d60a4040120049e48295adbd7c33e3e4; no voters:
I20251211 20:54:51.484892 6477 raft_consensus.cc:359] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.485082 6477 raft_consensus.cc:385] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.485103 6483 raft_consensus.cc:2804] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:54:51.485179 6477 raft_consensus.cc:740] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60a4040120049e48295adbd7c33e3e4, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.485267 6483 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:54:51.485428 6483 raft_consensus.cc:3060] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.485404 6477 consensus_queue.cc:260] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.489259 6483 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.490121 6483 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 election: Requested vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
W20251211 20:54:51.491451 6344 leader_election.cc:343] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:54:51.494169 6483 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.494365 6483 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.495028 6483 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:54:51.496377 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:54:51.497035 6480 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77: No bootstrap required, opened a new log
I20251211 20:54:51.497274 6480 ts_tablet_manager.cc:1403] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent bootstrapping tablet: real 0.049s user 0.013s sys 0.020s
I20251211 20:54:51.497656 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:54:51.498353 6480 raft_consensus.cc:359] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.498519 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77"
I20251211 20:54:51.498610 6480 raft_consensus.cc:385] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.498693 6480 raft_consensus.cc:740] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acae5cddb8e4a9b8777025ea93a4b77, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.498966 6480 consensus_queue.cc:260] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.499223 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:54:51.499555 6480 ts_tablet_manager.cc:1434] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:54:51.499528 6278 raft_consensus.cc:2468] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60a4040120049e48295adbd7c33e3e4 in term 0.
I20251211 20:54:51.499931 6480 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77: Bootstrap starting.
I20251211 20:54:51.502209 6480 tablet_bootstrap.cc:654] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77: Neither blocks nor log segments found. Creating new log.
W20251211 20:54:51.505528 6344 leader_election.cc:343] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:54:51.505685 6344 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60a4040120049e48295adbd7c33e3e4; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, cd482952c9fd4a1fb75c31f128338e89
I20251211 20:54:51.506050 6483 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20251211 20:54:51.506641 6344 leader_election.cc:343] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:54:51.507275 6344 leader_election.cc:343] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:54:51.507725 6344 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4; no voters:
I20251211 20:54:51.508036 6483 raft_consensus.cc:2804] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:54:51.508185 6483 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:54:51.508189 6484 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.508318 6483 raft_consensus.cc:3060] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.508383 6484 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.509553 6482 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.511725 6483 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.513204 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77"
I20251211 20:54:51.513381 6484 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.513434 6278 raft_consensus.cc:3060] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.517263 6477 ts_tablet_manager.cc:1434] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4: Time spent starting tablet: real 0.033s user 0.008s sys 0.016s
I20251211 20:54:51.517274 6278 raft_consensus.cc:2468] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60a4040120049e48295adbd7c33e3e4 in term 1.
I20251211 20:54:51.517633 6344 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4; no voters:
I20251211 20:54:51.517952 6496 raft_consensus.cc:2804] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:54:51.518021 6483 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 election: Requested vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:54:51.518235 6496 raft_consensus.cc:697] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 1 LEADER]: Becoming Leader. State: Replica: d60a4040120049e48295adbd7c33e3e4, State: Running, Role: LEADER
I20251211 20:54:51.518499 6482 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.519141 6482 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.519942 6480 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77: No bootstrap required, opened a new log
I20251211 20:54:51.520116 6480 ts_tablet_manager.cc:1403] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.006s
I20251211 20:54:51.519935 6496 consensus_queue.cc:237] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.521145 6480 raft_consensus.cc:359] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.521389 6480 raft_consensus.cc:385] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.521502 6480 raft_consensus.cc:740] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acae5cddb8e4a9b8777025ea93a4b77, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.521754 6480 consensus_queue.cc:260] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.522835 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89"
I20251211 20:54:51.524452 6147 raft_consensus.cc:3060] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.524863 6146 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89"
I20251211 20:54:51.525183 6146 raft_consensus.cc:3060] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.525411 6148 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89" is_pre_election: true
I20251211 20:54:51.529670 6147 raft_consensus.cc:2468] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60a4040120049e48295adbd7c33e3e4 in term 1.
I20251211 20:54:51.530128 6344 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89, d60a4040120049e48295adbd7c33e3e4; no voters: 4acae5cddb8e4a9b8777025ea93a4b77
I20251211 20:54:51.531970 6477 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4: Bootstrap starting.
I20251211 20:54:51.533080 6146 raft_consensus.cc:2468] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60a4040120049e48295adbd7c33e3e4 in term 1.
I20251211 20:54:51.533789 6483 raft_consensus.cc:2804] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:54:51.536269 6477 tablet_bootstrap.cc:654] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.536682 6483 raft_consensus.cc:697] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 1 LEADER]: Becoming Leader. State: Replica: d60a4040120049e48295adbd7c33e3e4, State: Running, Role: LEADER
I20251211 20:54:51.536983 6483 consensus_queue.cc:237] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.538435 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89" is_pre_election: true
I20251211 20:54:51.538738 6147 raft_consensus.cc:2468] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 0.
I20251211 20:54:51.538815 6480 ts_tablet_manager.cc:1434] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent starting tablet: real 0.019s user 0.003s sys 0.000s
I20251211 20:54:51.539286 6480 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77: Bootstrap starting.
I20251211 20:54:51.541558 6480 tablet_bootstrap.cc:654] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.544173 6477 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4: No bootstrap required, opened a new log
I20251211 20:54:51.544334 6484 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.544374 6477 ts_tablet_manager.cc:1403] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.004s
I20251211 20:54:51.544489 6484 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.545054 6484 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.545228 6477 raft_consensus.cc:359] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.545429 6477 raft_consensus.cc:385] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.545627 6477 raft_consensus.cc:740] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60a4040120049e48295adbd7c33e3e4, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.545887 6477 consensus_queue.cc:260] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.546455 6477 ts_tablet_manager.cc:1434] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:54:51.547187 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89" is_pre_election: true
I20251211 20:54:51.547961 6214 leader_election.cc:304] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, cd482952c9fd4a1fb75c31f128338e89; no voters:
I20251211 20:54:51.548435 6484 raft_consensus.cc:2804] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:54:51.548501 6477 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4: Bootstrap starting.
I20251211 20:54:51.548535 6484 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:54:51.548651 6484 raft_consensus.cc:3060] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Advancing to term 1
W20251211 20:54:51.548681 6214 leader_election.cc:343] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:54:51.550330 6477 tablet_bootstrap.cc:654] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.551789 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:54:51.552052 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:54:51.552162 6484 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.552364 6277 raft_consensus.cc:2393] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd482952c9fd4a1fb75c31f128338e89 in current term 1: Already voted for candidate d60a4040120049e48295adbd7c33e3e4 in this term.
I20251211 20:54:51.552515 6278 raft_consensus.cc:2393] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd482952c9fd4a1fb75c31f128338e89 in current term 1: Already voted for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in this term.
I20251211 20:54:51.553087 6484 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 election: Requested vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.553366 6476 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: No bootstrap required, opened a new log
I20251211 20:54:51.553555 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:54:51.553643 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89"
I20251211 20:54:51.553972 6147 raft_consensus.cc:3060] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.553922 6408 raft_consensus.cc:2468] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 0.
I20251211 20:54:51.554381 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:54:51.554670 6407 raft_consensus.cc:2468] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 0.
I20251211 20:54:51.555624 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:54:51.556445 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4"
I20251211 20:54:51.556591 6407 raft_consensus.cc:3060] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.556842 6147 raft_consensus.cc:2468] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 1.
I20251211 20:54:51.557121 6476 ts_tablet_manager.cc:1403] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 0.083s user 0.021s sys 0.037s
I20251211 20:54:51.557844 6476 raft_consensus.cc:359] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.558041 6476 raft_consensus.cc:385] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.558135 6476 raft_consensus.cc:740] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.558336 6476 consensus_queue.cc:260] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.558781 6476 ts_tablet_manager.cc:1434] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:54:51.559028 6476 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:54:51.559381 6407 raft_consensus.cc:2468] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 1.
I20251211 20:54:51.559753 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:54:51.560024 6408 raft_consensus.cc:2393] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd482952c9fd4a1fb75c31f128338e89 in current term 1: Already voted for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in this term.
I20251211 20:54:51.560456 6214 leader_election.cc:304] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, cd482952c9fd4a1fb75c31f128338e89; no voters:
I20251211 20:54:51.560791 6476 tablet_bootstrap.cc:654] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.561064 6214 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4; no voters: cd482952c9fd4a1fb75c31f128338e89
I20251211 20:54:51.561431 6484 raft_consensus.cc:2804] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:54:51.561158 6006 catalog_manager.cc:5654] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d60a4040120049e48295adbd7c33e3e4 (127.5.211.195). New cstate: current_term: 1 leader_uuid: "d60a4040120049e48295adbd7c33e3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } health_report { overall_health: HEALTHY } } }
I20251211 20:54:51.562371 6508 raft_consensus.cc:2804] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:54:51.562460 6508 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:54:51.562572 6508 raft_consensus.cc:3060] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.562610 6480 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77: No bootstrap required, opened a new log
I20251211 20:54:51.562819 6480 ts_tablet_manager.cc:1403] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent bootstrapping tablet: real 0.024s user 0.000s sys 0.003s
I20251211 20:54:51.563809 6480 raft_consensus.cc:359] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.564056 6480 raft_consensus.cc:385] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.564134 6480 raft_consensus.cc:740] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acae5cddb8e4a9b8777025ea93a4b77, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.564409 6480 consensus_queue.cc:260] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.565192 6480 ts_tablet_manager.cc:1434] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20251211 20:54:51.565657 6480 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77: Bootstrap starting.
I20251211 20:54:51.567421 6476 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: No bootstrap required, opened a new log
I20251211 20:54:51.567438 6480 tablet_bootstrap.cc:654] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.567646 6476 ts_tablet_manager.cc:1403] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20251211 20:54:51.568126 6484 raft_consensus.cc:697] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 LEADER]: Becoming Leader. State: Replica: 4acae5cddb8e4a9b8777025ea93a4b77, State: Running, Role: LEADER
I20251211 20:54:51.568361 6477 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4: No bootstrap required, opened a new log
I20251211 20:54:51.568496 6477 ts_tablet_manager.cc:1403] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4: Time spent bootstrapping tablet: real 0.020s user 0.010s sys 0.002s
I20251211 20:54:51.568500 6476 raft_consensus.cc:359] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.568544 6484 consensus_queue.cc:237] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.568676 6476 raft_consensus.cc:385] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.568737 6476 raft_consensus.cc:740] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.568975 6476 consensus_queue.cc:260] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.569200 6477 raft_consensus.cc:359] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.569360 6477 raft_consensus.cc:385] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.569445 6477 raft_consensus.cc:740] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d60a4040120049e48295adbd7c33e3e4, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.569509 6476 ts_tablet_manager.cc:1434] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:54:51.569761 6476 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:54:51.570324 6006 catalog_manager.cc:5654] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d60a4040120049e48295adbd7c33e3e4 (127.5.211.195). New cstate: current_term: 1 leader_uuid: "d60a4040120049e48295adbd7c33e3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } health_report { overall_health: HEALTHY } } }
W20251211 20:54:51.570575 6324 tablet.cc:2378] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:54:51.570569 6084 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:54:51.571280 6482 raft_consensus.cc:2749] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:54:51.571483 6476 tablet_bootstrap.cc:654] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: Neither blocks nor log segments found. Creating new log.
I20251211 20:54:51.572099 6084 leader_election.cc:304] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:54:51.572455 6482 raft_consensus.cc:2749] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:54:51.574535 6477 consensus_queue.cc:260] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.576188 6477 ts_tablet_manager.cc:1434] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20251211 20:54:51.576452 6510 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.576606 6510 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.577586 6510 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.578142 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89" is_pre_election: true
I20251211 20:54:51.578418 6147 raft_consensus.cc:2393] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4acae5cddb8e4a9b8777025ea93a4b77 in current term 1: Already voted for candidate d60a4040120049e48295adbd7c33e3e4 in this term.
I20251211 20:54:51.579169 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:54:51.579927 6214 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77; no voters: cd482952c9fd4a1fb75c31f128338e89, d60a4040120049e48295adbd7c33e3e4
I20251211 20:54:51.580493 6510 raft_consensus.cc:3060] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.581921 6480 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77: No bootstrap required, opened a new log
I20251211 20:54:51.582101 6480 ts_tablet_manager.cc:1403] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.000s
I20251211 20:54:51.582882 6480 raft_consensus.cc:359] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.583078 6480 raft_consensus.cc:385] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.583169 6480 raft_consensus.cc:740] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acae5cddb8e4a9b8777025ea93a4b77, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.583487 6510 raft_consensus.cc:2749] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20251211 20:54:51.583398 6480 consensus_queue.cc:260] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.583659 6510 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.583803 6510 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.583992 6480 ts_tablet_manager.cc:1434] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20251211 20:54:51.584304 6510 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.584893 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89" is_pre_election: true
I20251211 20:54:51.585001 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:54:51.585180 6408 raft_consensus.cc:2468] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 0.
W20251211 20:54:51.585429 6214 leader_election.cc:343] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:54:51.585716 6214 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4; no voters: cd482952c9fd4a1fb75c31f128338e89
I20251211 20:54:51.585918 6508 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.586081 6510 raft_consensus.cc:2804] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:54:51.586172 6510 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:54:51.586301 6510 raft_consensus.cc:3060] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.586573 6508 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 election: Requested vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.587165 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89"
I20251211 20:54:51.587240 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4"
I20251211 20:54:51.587355 6147 raft_consensus.cc:3060] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.587424 6408 raft_consensus.cc:3060] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.589545 6476 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: No bootstrap required, opened a new log
I20251211 20:54:51.589651 6476 ts_tablet_manager.cc:1403] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 0.020s user 0.005s sys 0.001s
I20251211 20:54:51.589816 6147 raft_consensus.cc:2468] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 1.
I20251211 20:54:51.590018 6408 raft_consensus.cc:2468] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 1.
I20251211 20:54:51.589790 6004 catalog_manager.cc:5654] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194). New cstate: current_term: 1 leader_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20251211 20:54:51.590394 6476 raft_consensus.cc:359] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.590553 6476 raft_consensus.cc:385] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:54:51.590649 6476 raft_consensus.cc:740] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:54:51.590869 6476 consensus_queue.cc:260] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.591357 6476 ts_tablet_manager.cc:1434] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:54:51.591987 6510 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.592550 6510 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 election: Requested vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.593704 6214 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4; no voters:
I20251211 20:54:51.593727 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89"
I20251211 20:54:51.593863 6147 raft_consensus.cc:3060] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.594064 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4"
I20251211 20:54:51.594338 6408 raft_consensus.cc:3060] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.594934 6510 raft_consensus.cc:2804] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:54:51.595094 6510 raft_consensus.cc:697] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 LEADER]: Becoming Leader. State: Replica: 4acae5cddb8e4a9b8777025ea93a4b77, State: Running, Role: LEADER
I20251211 20:54:51.595350 6510 consensus_queue.cc:237] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.596387 6147 raft_consensus.cc:2468] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 1.
I20251211 20:54:51.597630 6214 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, cd482952c9fd4a1fb75c31f128338e89; no voters:
I20251211 20:54:51.597918 6510 raft_consensus.cc:2804] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:54:51.598093 6510 raft_consensus.cc:697] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 LEADER]: Becoming Leader. State: Replica: 4acae5cddb8e4a9b8777025ea93a4b77, State: Running, Role: LEADER
I20251211 20:54:51.598487 6408 raft_consensus.cc:2468] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acae5cddb8e4a9b8777025ea93a4b77 in term 1.
I20251211 20:54:51.598402 6510 consensus_queue.cc:237] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.599967 6004 catalog_manager.cc:5654] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194). New cstate: current_term: 1 leader_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20251211 20:54:51.604651 6004 catalog_manager.cc:5654] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194). New cstate: current_term: 1 leader_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } health_report { overall_health: UNKNOWN } } }
I20251211 20:54:51.615978 6483 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.616158 6483 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.616732 6483 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:54:51.617089 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89" is_pre_election: true
I20251211 20:54:51.617178 6482 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:54:51.617275 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:54:51.617377 6482 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.617514 6278 raft_consensus.cc:2468] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60a4040120049e48295adbd7c33e3e4 in term 0.
I20251211 20:54:51.617662 6147 raft_consensus.cc:2468] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d60a4040120049e48295adbd7c33e3e4 in term 0.
I20251211 20:54:51.617961 6344 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4; no voters:
I20251211 20:54:51.618036 6482 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.618320 6483 raft_consensus.cc:2804] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:54:51.618435 6483 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:54:51.618573 6483 raft_consensus.cc:3060] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.618541 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:54:51.618721 6278 raft_consensus.cc:2468] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cd482952c9fd4a1fb75c31f128338e89 in term 0.
I20251211 20:54:51.618813 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:54:51.619155 6084 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, cd482952c9fd4a1fb75c31f128338e89; no voters:
I20251211 20:54:51.619452 6482 raft_consensus.cc:2804] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:54:51.619560 6482 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:54:51.619647 6482 raft_consensus.cc:3060] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.620939 6496 consensus_queue.cc:1048] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.621022 6483 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.621357 6408 raft_consensus.cc:2393] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cd482952c9fd4a1fb75c31f128338e89 in current term 1: Already voted for candidate d60a4040120049e48295adbd7c33e3e4 in this term.
I20251211 20:54:51.621652 6483 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 election: Requested vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:54:51.622205 6482 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.622262 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77"
I20251211 20:54:51.622457 6277 raft_consensus.cc:3060] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:54:51.622419 6147 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "d60a4040120049e48295adbd7c33e3e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd482952c9fd4a1fb75c31f128338e89"
I20251211 20:54:51.622660 6147 raft_consensus.cc:2393] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d60a4040120049e48295adbd7c33e3e4 in current term 1: Already voted for candidate cd482952c9fd4a1fb75c31f128338e89 in this term.
I20251211 20:54:51.623169 6482 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 1 election: Requested vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:51.625411 6277 raft_consensus.cc:2468] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d60a4040120049e48295adbd7c33e3e4 in term 1.
I20251211 20:54:51.626704 6344 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4; no voters: cd482952c9fd4a1fb75c31f128338e89
I20251211 20:54:51.627441 6483 raft_consensus.cc:2804] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:54:51.627550 6496 consensus_queue.cc:1048] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:54:51.628508 6194 tablet.cc:2378] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:54:51.631524 6496 consensus_queue.cc:1048] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.633339 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77"
I20251211 20:54:51.633608 6277 raft_consensus.cc:2393] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cd482952c9fd4a1fb75c31f128338e89 in current term 1: Already voted for candidate d60a4040120049e48295adbd7c33e3e4 in this term.
I20251211 20:54:51.634720 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4"
I20251211 20:54:51.636747 6496 consensus_queue.cc:1048] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.642120 6483 raft_consensus.cc:697] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 1 LEADER]: Becoming Leader. State: Replica: d60a4040120049e48295adbd7c33e3e4, State: Running, Role: LEADER
I20251211 20:54:51.642488 6483 consensus_queue.cc:237] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:51.650892 6084 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:54:51.651285 6482 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20251211 20:54:51.652128 6004 catalog_manager.cc:5654] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to d60a4040120049e48295adbd7c33e3e4 (127.5.211.195). New cstate: current_term: 1 leader_uuid: "d60a4040120049e48295adbd7c33e3e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } health_report { overall_health: HEALTHY } } }
I20251211 20:54:51.676209 6510 consensus_queue.cc:1048] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.680660 6510 consensus_queue.cc:1048] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.696576 6510 consensus_queue.cc:1048] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.700960 6510 consensus_queue.cc:1048] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.701064 6508 consensus_queue.cc:1048] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.704013 6508 consensus_queue.cc:1048] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.713457 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.196:0
--local_ip_for_outbound_sockets=127.5.211.196
--webserver_interface=127.5.211.196
--webserver_port=0
--tserver_master_addrs=127.5.211.254:40613
--builtin_ntp_servers=127.5.211.212:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--follower_unavailable_considered_failed_sec=2
--enable_log_gc=false with env {}
I20251211 20:54:51.719820 6503 consensus_queue.cc:1048] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:54:51.744911 6503 consensus_queue.cc:1048] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:54:52.666679 6538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:54:52.667140 6538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:54:52.667243 6538 flags.cc:425] Enabled unsafe flag: --enable_log_gc=false
W20251211 20:54:52.667332 6538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:54:52.685047 6538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:54:52.685410 6538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.196
I20251211 20:54:52.699968 6538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=2
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.5.211.196
--webserver_port=0
--enable_log_gc=false
--tserver_master_addrs=127.5.211.254:40613
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.196
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:54:52.702061 6538 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:54:52.704315 6538 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:54:52.739555 6578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:54:52.739704 6579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:54:52.795413 6538 server_base.cc:1047] running on GCE node
W20251211 20:54:52.799484 6581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:54:52.800513 6538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:54:52.867266 6538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:54:52.868834 6538 hybrid_clock.cc:648] HybridClock initialized: now 1765486492868737 us; error 73 us; skew 500 ppm
I20251211 20:54:52.869256 6538 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:54:52.912050 6538 webserver.cc:492] Webserver started at http://127.5.211.196:33641/ using document root <none> and password file <none>
I20251211 20:54:52.914215 6538 fs_manager.cc:362] Metadata directory not provided
I20251211 20:54:52.914507 6538 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:54:52.915062 6538 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:54:52.918663 6538 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/data/instance:
uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41"
format_stamp: "Formatted at 2025-12-11 20:54:52 on dist-test-slave-sk6m"
I20251211 20:54:52.919813 6538 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/wal/instance:
uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41"
format_stamp: "Formatted at 2025-12-11 20:54:52 on dist-test-slave-sk6m"
I20251211 20:54:52.937240 6538 fs_manager.cc:696] Time spent creating directory manager: real 0.017s user 0.001s sys 0.004s
I20251211 20:54:52.942595 6587 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:52.949358 6538 fs_manager.cc:730] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20251211 20:54:52.949666 6538 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/wal
uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41"
format_stamp: "Formatted at 2025-12-11 20:54:52 on dist-test-slave-sk6m"
I20251211 20:54:52.950060 6538 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:54:53.028414 6538 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:54:53.029632 6538 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:54:53.030045 6538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:54:53.032565 6538 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:54:53.049291 6538 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:54:53.049464 6538 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20251211 20:54:53.049633 6538 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:54:53.049738 6538 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:54:54.902621 6538 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.196:35667
I20251211 20:54:54.902639 6702 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.196:35667 every 8 connection(s)
I20251211 20:54:54.904721 6538 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
I20251211 20:54:54.905575 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 6538
I20251211 20:54:54.905756 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-3/wal/instance
I20251211 20:54:55.005532 6703 heartbeater.cc:344] Connected to a master server at 127.5.211.254:40613
I20251211 20:54:55.005955 6703 heartbeater.cc:461] Registering TS with master...
I20251211 20:54:55.007156 6703 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:54:55.008736 6003 ts_manager.cc:194] Registered new tserver with Master: 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667)
I20251211 20:54:55.009797 6003 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.196:54343
I20251211 20:54:55.472856 6003 ts_manager.cc:295] Set tserver state for cd482952c9fd4a1fb75c31f128338e89 to MAINTENANCE_MODE
I20251211 20:54:55.474036 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 6066
I20251211 20:54:55.475375 6188 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:54:55.841885 6721 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:54:55.842741 6721 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:54:55.843904 6721 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:54:55.870497 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 6066
W20251211 20:54:56.011164 5991 connection.cc:537] server connection from 127.5.211.193:44285 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:54:56.013188 6703 heartbeater.cc:499] Master 127.5.211.254:40613 was elected leader, sending a full tablet report...
W20251211 20:54:56.014181 6344 connection.cc:537] client connection to 127.5.211.193:34639 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:54:56.014619 6344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:54:56.012292 6214 connection.cc:537] client connection to 127.5.211.193:34639 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:54:56.015733 6214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:54:56.026968 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:54:56.032107 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:54:56.032307 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:54:56.032428 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:54:56.035851 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:54:56.036788 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:54:56.369369 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:54:56.449898 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:54:56.457868 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:54:56.458693 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:54:56.494931 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:54:56.495146 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:54:56.859331 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:54:56.946659 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:54:56.947008 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:54:56.997252 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:54:57.042126 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:54:57.068670 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:54:57.374082 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:54:57.462790 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:54:57.477056 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:54:57.487454 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:54:57.572907 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:54:57.588630 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20251211 20:54:57.891170 6707 consensus_queue.cc:579] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.036s)
I20251211 20:54:57.905795 6520 consensus_queue.cc:579] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.077s)
I20251211 20:54:57.909158 6508 consensus_queue.cc:579] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.057s)
W20251211 20:54:57.916676 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20251211 20:54:57.927373 6508 consensus_queue.cc:579] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.075s)
I20251211 20:54:57.937335 6576 consensus_queue.cc:579] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.085s)
W20251211 20:54:57.970623 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20251211 20:54:57.975734 6571 consensus_queue.cc:579] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.121s)
W20251211 20:54:57.983274 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:54:58.057653 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:54:58.147946 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:54:58.152500 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:54:58.439158 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20251211 20:54:58.465265 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20251211 20:54:58.517541 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20251211 20:54:58.569449 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20251211 20:54:58.604772 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20251211 20:54:58.676955 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20251211 20:54:58.973672 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20251211 20:54:58.973863 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20251211 20:54:58.990026 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20251211 20:54:59.067260 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20251211 20:54:59.137787 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 5975
I20251211 20:54:59.139531 6036 generic_service.cc:196] Checking for leaks (request via RPC)
W20251211 20:54:59.143388 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20251211 20:54:59.224467 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20251211 20:54:59.424582 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 5975
I20251211 20:54:59.456488 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.211.254:40613
--webserver_interface=127.5.211.254
--webserver_port=37167
--builtin_ntp_servers=127.5.211.212:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.211.254:40613 with env {}
W20251211 20:54:59.464273 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20251211 20:54:59.488850 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20251211 20:54:59.550676 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20251211 20:54:59.585008 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20251211 20:54:59.714299 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20251211 20:54:59.724277 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20251211 20:54:59.952854 6323 heartbeater.cc:646] Failed to heartbeat to 127.5.211.254:40613 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.211.254:40613: connect: Connection refused (error 111)
W20251211 20:54:59.966720 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20251211 20:54:59.993150 6453 heartbeater.cc:646] Failed to heartbeat to 127.5.211.254:40613 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.211.254:40613: connect: Connection refused (error 111)
W20251211 20:55:00.022379 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20251211 20:55:00.062572 6703 heartbeater.cc:646] Failed to heartbeat to 127.5.211.254:40613 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.211.254:40613: connect: Connection refused (error 111)
W20251211 20:55:00.102947 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20251211 20:55:00.140956 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20251211 20:55:00.219838 6742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:00.220358 6742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:00.220551 6742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:00.235679 6742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251211 20:55:00.235864 6742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:00.235975 6742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251211 20:55:00.236080 6742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251211 20:55:00.251592 6742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.211.254:40613
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.211.254:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.211.254
--webserver_port=37167
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:00.255654 6742 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:00.259853 6742 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:00.289551 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20251211 20:55:00.347368 6748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:00.392527 6749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:00.446066 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20251211 20:55:00.464323 6751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:00.500519 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20251211 20:55:00.518220 6742 server_base.cc:1047] running on GCE node
I20251211 20:55:00.519876 6742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:00.527139 6742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:00.537292 6742 hybrid_clock.cc:648] HybridClock initialized: now 1765486500537176 us; error 95 us; skew 500 ppm
I20251211 20:55:00.538057 6742 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:00.563189 6742 webserver.cc:492] Webserver started at http://127.5.211.254:37167/ using document root <none> and password file <none>
I20251211 20:55:00.564435 6742 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:00.564800 6742 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:00.580399 6742 fs_manager.cc:714] Time spent opening directory manager: real 0.012s user 0.007s sys 0.004s
I20251211 20:55:00.600512 6758 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:00.609771 6742 fs_manager.cc:730] Time spent opening block manager: real 0.027s user 0.009s sys 0.007s
I20251211 20:55:00.609951 6742 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
uuid: "e4d4d4e72e654afe9bf24bdad38cef52"
format_stamp: "Formatted at 2025-12-11 20:54:49 on dist-test-slave-sk6m"
I20251211 20:55:00.610942 6742 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20251211 20:55:00.632946 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20251211 20:55:00.665351 6742 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
W20251211 20:55:00.665717 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20251211 20:55:00.666785 6742 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:00.667204 6742 kserver.cc:163] Server-wide thread pool size limit: 3276
W20251211 20:55:00.702150 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20251211 20:55:00.830509 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20251211 20:55:00.904068 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20251211 20:55:01.010583 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20251211 20:55:01.028167 6344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111) [suppressed 143 similar messages]
W20251211 20:55:01.142930 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20251211 20:55:01.147306 6742 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.254:40613
I20251211 20:55:01.147306 6814 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.254:40613 every 8 connection(s)
I20251211 20:55:01.149369 6742 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/master-0/data/info.pb
W20251211 20:55:01.149626 6214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111) [suppressed 146 similar messages]
I20251211 20:55:01.155072 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 6742
I20251211 20:55:01.169317 6815 sys_catalog.cc:263] Verifying existing consensus state
I20251211 20:55:01.173903 6815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: Bootstrap starting.
I20251211 20:55:01.197506 6815 log.cc:826] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: Log is configured to *not* fsync() on all Append() calls
W20251211 20:55:01.211380 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20251211 20:55:01.263161 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20251211 20:55:01.277503 6815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: Bootstrap replayed 1/1 log segments. Stats: ops{read=13 overwritten=0 applied=13 ignored=0} inserts{seen=11 ignored=0} mutations{seen=12 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:55:01.278227 6815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: Bootstrap complete.
I20251211 20:55:01.289289 6815 raft_consensus.cc:359] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } }
I20251211 20:55:01.290324 6815 raft_consensus.cc:740] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4d4d4e72e654afe9bf24bdad38cef52, State: Initialized, Role: FOLLOWER
I20251211 20:55:01.291515 6815 consensus_queue.cc:260] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } }
I20251211 20:55:01.291858 6815 raft_consensus.cc:399] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251211 20:55:01.292039 6815 raft_consensus.cc:493] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251211 20:55:01.292281 6815 raft_consensus.cc:3060] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 1 FOLLOWER]: Advancing to term 2
I20251211 20:55:01.299763 6815 raft_consensus.cc:515] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } }
I20251211 20:55:01.300580 6815 leader_election.cc:304] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e4d4d4e72e654afe9bf24bdad38cef52; no voters:
I20251211 20:55:01.314322 6815 leader_election.cc:290] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [CANDIDATE]: Term 2 election: Requested vote from peers
I20251211 20:55:01.314440 6818 raft_consensus.cc:2804] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 2 FOLLOWER]: Leader election won for term 2
I20251211 20:55:01.322487 6818 raft_consensus.cc:697] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [term 2 LEADER]: Becoming Leader. State: Replica: e4d4d4e72e654afe9bf24bdad38cef52, State: Running, Role: LEADER
I20251211 20:55:01.323158 6818 consensus_queue.cc:237] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 1.13, Last appended by leader: 13, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } }
I20251211 20:55:01.323689 6815 sys_catalog.cc:565] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: configured and running, proceeding with master startup.
I20251211 20:55:01.349025 6820 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e4d4d4e72e654afe9bf24bdad38cef52. Latest consensus state: current_term: 2 leader_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } } }
I20251211 20:55:01.349624 6820 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: This master's current role is: LEADER
I20251211 20:55:01.349052 6819 sys_catalog.cc:455] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e4d4d4e72e654afe9bf24bdad38cef52" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 40613 } } }
I20251211 20:55:01.350503 6819 sys_catalog.cc:458] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52 [sys.catalog]: This master's current role is: LEADER
I20251211 20:55:01.382926 6825 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
W20251211 20:55:01.486446 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20251211 20:55:01.507099 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20251211 20:55:01.574285 6825 catalog_manager.cc:679] Loaded metadata for table test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3]
I20251211 20:55:01.581157 6825 tablet_loader.cc:96] loaded metadata for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9 (table test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3])
I20251211 20:55:01.581835 6825 tablet_loader.cc:96] loaded metadata for tablet 4927adf430d04b6199dbbfaf1de1c16d (table test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3])
I20251211 20:55:01.586959 6825 tablet_loader.cc:96] loaded metadata for tablet a9dbb37290964ca7ba650d2de1404245 (table test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3])
I20251211 20:55:01.587563 6825 tablet_loader.cc:96] loaded metadata for tablet ce1e22ac4b814a508b20688b15d0c959 (table test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3])
W20251211 20:55:01.612905 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20251211 20:55:01.619908 6825 tablet_loader.cc:96] loaded metadata for tablet d915f836ed2c479c9d6623e11266ed55 (table test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3])
I20251211 20:55:01.620422 6825 tablet_loader.cc:96] loaded metadata for tablet e7ef1dab1f4c47379f2756f162302ecc (table test-workload [id=6ac61f59681841d3ab07d5f187a8c6f3])
I20251211 20:55:01.620795 6825 catalog_manager.cc:1494] Initializing Kudu cluster ID...
W20251211 20:55:01.628669 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20251211 20:55:01.649338 6825 catalog_manager.cc:1269] Loaded cluster ID: 533ab04ce03b4bf4b47ded938756be0f
I20251211 20:55:01.649803 6825 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251211 20:55:01.701614 6825 catalog_manager.cc:1514] Loading token signing keys...
I20251211 20:55:01.723536 6825 catalog_manager.cc:6038] T 00000000000000000000000000000000 P e4d4d4e72e654afe9bf24bdad38cef52: Loaded TSK: 0
I20251211 20:55:01.734403 6825 catalog_manager.cc:1524] Initializing in-progress tserver states...
W20251211 20:55:01.788465 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20251211 20:55:01.824319 6835 catalog_manager.cc:805] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20251211 20:55:01.861559 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20251211 20:55:02.035710 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20251211 20:55:02.079658 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20251211 20:55:02.102607 6453 heartbeater.cc:344] Connected to a master server at 127.5.211.254:40613
I20251211 20:55:02.105405 6776 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" instance_seqno: 1765486491220963) as {username='slave'} at 127.5.211.195:49427; Asking this server to re-register.
I20251211 20:55:02.107381 6323 heartbeater.cc:344] Connected to a master server at 127.5.211.254:40613
I20251211 20:55:02.107982 6453 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:02.108240 6453 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:55:02.113430 6776 ts_manager.cc:194] Registered new tserver with Master: d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:02.122514 6775 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" instance_seqno: 1765486490809493) as {username='slave'} at 127.5.211.194:50357; Asking this server to re-register.
W20251211 20:55:02.126567 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20251211 20:55:02.133587 6323 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:02.133906 6323 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
W20251211 20:55:02.170157 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
I20251211 20:55:02.174948 6703 heartbeater.cc:344] Connected to a master server at 127.5.211.254:40613
I20251211 20:55:02.176671 6775 master_service.cc:438] Got heartbeat from unknown tserver (permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" instance_seqno: 1765486494693193) as {username='slave'} at 127.5.211.196:56331; Asking this server to re-register.
I20251211 20:55:02.180606 6703 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:02.180855 6703 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:55:02.182085 6775 ts_manager.cc:194] Registered new tserver with Master: 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667)
I20251211 20:55:02.200879 6776 ts_manager.cc:194] Registered new tserver with Master: 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
W20251211 20:55:02.295059 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20251211 20:55:02.417536 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20251211 20:55:02.591545 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20251211 20:55:02.608608 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20251211 20:55:02.610613 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 61: this message will repeat every 5th retry.
W20251211 20:55:02.704754 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20251211 20:55:02.864243 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20251211 20:55:02.945753 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20251211 20:55:03.123838 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20251211 20:55:03.153805 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 66: this message will repeat every 5th retry.
W20251211 20:55:03.200359 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20251211 20:55:03.242125 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20251211 20:55:03.523488 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20251211 20:55:03.531087 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20251211 20:55:03.644345 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20251211 20:55:03.724927 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20251211 20:55:03.734175 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20251211 20:55:03.757658 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 71: this message will repeat every 5th retry.
W20251211 20:55:04.037357 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20251211 20:55:04.076898 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20251211 20:55:04.222357 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20251211 20:55:04.242832 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20251211 20:55:04.317121 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20251211 20:55:04.345499 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.193:34639
--local_ip_for_outbound_sockets=127.5.211.193
--tserver_master_addrs=127.5.211.254:40613
--webserver_port=37369
--webserver_interface=127.5.211.193
--builtin_ntp_servers=127.5.211.212:38545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--follower_unavailable_considered_failed_sec=2
--enable_log_gc=false with env {}
W20251211 20:55:04.381855 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 76: this message will repeat every 5th retry.
W20251211 20:55:04.585429 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20251211 20:55:04.751654 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20251211 20:55:04.760782 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20251211 20:55:04.838125 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20251211 20:55:04.855139 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
W20251211 20:55:04.967020 6852 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:04.967454 6852 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:04.967614 6852 flags.cc:425] Enabled unsafe flag: --enable_log_gc=false
W20251211 20:55:04.967754 6852 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:04.981055 6852 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:04.981369 6852 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.193
I20251211 20:55:04.996897 6852 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:38545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--follower_unavailable_considered_failed_sec=2
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.193:34639
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.211.193
--webserver_port=37369
--enable_log_gc=false
--tserver_master_addrs=127.5.211.254:40613
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.193
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:04.999147 6852 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:05.001327 6852 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:05.028177 6857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:05.032016 6858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:05.053035 6861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:05.055773 6852 server_base.cc:1047] running on GCE node
I20251211 20:55:05.056866 6852 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
W20251211 20:55:05.066463 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 81: this message will repeat every 5th retry.
I20251211 20:55:05.077257 6852 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:05.109580 6852 hybrid_clock.cc:648] HybridClock initialized: now 1765486505102055 us; error 7475 us; skew 500 ppm
I20251211 20:55:05.110033 6852 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:05.127578 6852 webserver.cc:492] Webserver started at http://127.5.211.193:37369/ using document root <none> and password file <none>
I20251211 20:55:05.128677 6852 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:05.131244 6852 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:05.183768 6852 fs_manager.cc:714] Time spent opening directory manager: real 0.045s user 0.007s sys 0.004s
W20251211 20:55:05.202972 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20251211 20:55:05.268785 6866 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:05.274346 6852 fs_manager.cc:730] Time spent opening block manager: real 0.082s user 0.014s sys 0.025s
I20251211 20:55:05.278839 6852 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
uuid: "cd482952c9fd4a1fb75c31f128338e89"
format_stamp: "Formatted at 2025-12-11 20:54:50 on dist-test-slave-sk6m"
I20251211 20:55:05.280622 6852 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20251211 20:55:05.296012 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20251211 20:55:05.321907 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20251211 20:55:05.387681 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
W20251211 20:55:05.393529 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20251211 20:55:05.444638 6852 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:05.445775 6852 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:05.446244 6852 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:05.568948 6852 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:05.584136 6875 ts_tablet_manager.cc:548] Loading tablet metadata (0/6 complete)
W20251211 20:55:05.624953 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 86: this message will repeat every 5th retry.
I20251211 20:55:05.627527 6852 ts_tablet_manager.cc:585] Loaded tablet metadata (6 total tablets, 6 live tablets)
I20251211 20:55:05.627652 6852 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.050s user 0.007s sys 0.000s
I20251211 20:55:05.627775 6852 ts_tablet_manager.cc:600] Registering tablets (0/6 complete)
I20251211 20:55:05.632315 6875 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:55:05.651697 6852 ts_tablet_manager.cc:616] Registered 6 tablets
I20251211 20:55:05.651882 6852 ts_tablet_manager.cc:595] Time spent register tablets: real 0.024s user 0.015s sys 0.000s
W20251211 20:55:05.742779 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20251211 20:55:05.774644 6875 log.cc:826] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Log is configured to *not* fsync() on all Append() calls
W20251211 20:55:06.047104 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
I20251211 20:55:06.065315 6875 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap replayed 1/1 log segments. Stats: ops{read=181 overwritten=0 applied=161 ignored=0} inserts{seen=1304 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 20 replicates
I20251211 20:55:06.066215 6875 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap complete.
I20251211 20:55:06.068882 6875 ts_tablet_manager.cc:1403] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 0.437s user 0.257s sys 0.033s
I20251211 20:55:06.075812 6875 raft_consensus.cc:359] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Replica starting. Triggering 20 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:06.092378 6875 raft_consensus.cc:740] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:55:06.093508 6875 consensus_queue.cc:260] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 161, Last appended: 1.181, Last appended by leader: 181, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
W20251211 20:55:06.100533 6344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111) [suppressed 130 similar messages]
I20251211 20:55:06.100903 6875 ts_tablet_manager.cc:1434] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.032s user 0.021s sys 0.007s
W20251211 20:55:06.101454 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
I20251211 20:55:06.101665 6875 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
W20251211 20:55:06.137578 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20251211 20:55:06.137696 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20251211 20:55:06.235605 6214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111) [suppressed 131 similar messages]
W20251211 20:55:06.388888 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 91: this message will repeat every 5th retry.
W20251211 20:55:06.391131 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
I20251211 20:55:06.436355 6898 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:06.436631 6898 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:06.439332 6898 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:06.489910 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 181 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:06.506198 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 181 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:06.508427 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:06.509344 6898 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:55:06.522691 6214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.211.195:46117, user_credentials={real_user=slave}} blocked reactor thread for 52526.6us
W20251211 20:55:06.725054 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
I20251211 20:55:06.768415 6875 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap replayed 1/1 log segments. Stats: ops{read=160 overwritten=0 applied=160 ignored=0} inserts{seen=1315 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:55:06.769186 6875 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap complete.
I20251211 20:55:06.770484 6875 ts_tablet_manager.cc:1403] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 0.669s user 0.271s sys 0.054s
I20251211 20:55:06.771766 6875 raft_consensus.cc:359] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:06.772092 6875 raft_consensus.cc:740] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:55:06.772441 6875 consensus_queue.cc:260] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 160, Last appended: 1.160, Last appended by leader: 160, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:06.773116 6875 ts_tablet_manager.cc:1434] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20251211 20:55:06.773430 6875 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:55:06.844266 6898 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:06.844449 6898 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:06.845131 6898 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:06.846237 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 181 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:06.846352 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 181 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:06.846993 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:06.847393 6898 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:55:06.864482 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
W20251211 20:55:06.892668 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 101: this message will repeat every 5th retry.
W20251211 20:55:06.897773 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 101: this message will repeat every 5th retry.
I20251211 20:55:07.066171 6898 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:07.066334 6898 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:07.067258 6898 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:07.067775 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:07.067876 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:07.068397 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:07.068745 6898 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:55:07.133772 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 101: this message will repeat every 5th retry.
W20251211 20:55:07.258064 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 96: this message will repeat every 5th retry.
I20251211 20:55:07.298724 6898 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:07.298900 6898 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:07.299475 6898 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:07.300254 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 181 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:07.300319 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 181 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:07.301569 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:07.302116 6898 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:55:07.306691 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 101: this message will repeat every 5th retry.
I20251211 20:55:07.335325 6875 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap replayed 1/1 log segments. Stats: ops{read=160 overwritten=0 applied=159 ignored=0} inserts{seen=1331 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251211 20:55:07.346495 6875 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap complete.
I20251211 20:55:07.347913 6875 ts_tablet_manager.cc:1403] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 0.575s user 0.291s sys 0.087s
I20251211 20:55:07.349047 6875 raft_consensus.cc:359] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:07.365571 6875 raft_consensus.cc:740] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:55:07.365952 6875 consensus_queue.cc:260] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 159, Last appended: 1.160, Last appended by leader: 160, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:07.366493 6875 ts_tablet_manager.cc:1434] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.018s user 0.002s sys 0.000s
I20251211 20:55:07.366804 6875 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:55:07.412415 6898 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:07.412595 6898 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:07.413267 6898 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:07.413793 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:07.416296 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:07.419322 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:07.419816 6898 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:55:07.481982 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 106: this message will repeat every 5th retry.
W20251211 20:55:07.622186 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 106: this message will repeat every 5th retry.
W20251211 20:55:07.631167 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 101: this message will repeat every 5th retry.
I20251211 20:55:07.638046 6898 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:07.638219 6898 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:07.638870 6898 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:07.643157 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:07.646116 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:07.650334 6872 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:07.650844 6898 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:07.690131 6898 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:07.690351 6898 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:07.691274 6898 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:07.693436 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 181 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:07.694757 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 181 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:07.698802 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:07.699326 6898 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:55:07.731415 6214 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.211.195:46117, user_credentials={real_user=slave}} blocked reactor thread for 53712.1us
W20251211 20:55:07.837227 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 101: this message will repeat every 5th retry.
W20251211 20:55:07.889926 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 106: this message will repeat every 5th retry.
W20251211 20:55:07.893702 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 106: this message will repeat every 5th retry.
I20251211 20:55:07.913831 6898 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:07.914004 6898 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:07.914691 6898 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:07.915311 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:07.918474 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:07.919476 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:07.922168 6898 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:08.087559 6898 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:08.087744 6898 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:08.088418 6898 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:08.089076 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:08.098853 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:08.099450 6872 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:08.099865 6898 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:08.248276 6852 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.193:34639
I20251211 20:55:08.248335 7004 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.193:34639 every 8 connection(s)
I20251211 20:55:08.250329 6852 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
I20251211 20:55:08.261057 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 6852
I20251211 20:55:08.432266 7006 heartbeater.cc:344] Connected to a master server at 127.5.211.254:40613
I20251211 20:55:08.432639 7006 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:08.435070 7006 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:55:08.438660 6776 ts_manager.cc:194] Registered new tserver with Master: cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639)
I20251211 20:55:08.442128 6776 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.193:60317
W20251211 20:55:08.549512 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 111: this message will repeat every 5th retry.
W20251211 20:55:08.572764 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 111: this message will repeat every 5th retry.
W20251211 20:55:08.592779 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 106: this message will repeat every 5th retry.
I20251211 20:55:08.667354 6946 raft_consensus.cc:1217] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Deduplicated request from leader. Original: 1.160->[1.161-1.884] Dedup: 1.181->[1.182-1.884]
I20251211 20:55:08.683614 7006 heartbeater.cc:499] Master 127.5.211.254:40613 was elected leader, sending a full tablet report...
I20251211 20:55:08.688760 6875 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Bootstrap replayed 1/1 log segments. Stats: ops{read=160 overwritten=0 applied=159 ignored=0} inserts{seen=1309 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251211 20:55:08.689440 6875 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Bootstrap complete.
I20251211 20:55:08.690555 6875 ts_tablet_manager.cc:1403] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 1.324s user 0.253s sys 0.020s
I20251211 20:55:08.691282 6875 raft_consensus.cc:359] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:08.691818 6875 raft_consensus.cc:740] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:55:08.692615 6875 consensus_queue.cc:260] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 159, Last appended: 1.160, Last appended by leader: 160, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:08.693182 6875 ts_tablet_manager.cc:1434] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20251211 20:55:08.693739 6875 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
W20251211 20:55:09.664444 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 111: this message will repeat every 5th retry.
W20251211 20:55:10.090598 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 116: this message will repeat every 5th retry.
I20251211 20:55:10.135633 6875 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap replayed 1/1 log segments. Stats: ops{read=160 overwritten=0 applied=159 ignored=0} inserts{seen=1374 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251211 20:55:10.136293 6875 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Bootstrap complete.
I20251211 20:55:10.138623 6875 ts_tablet_manager.cc:1403] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 1.445s user 0.243s sys 0.054s
I20251211 20:55:10.139335 6875 raft_consensus.cc:359] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:10.140174 6875 raft_consensus.cc:740] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:55:10.144610 6875 consensus_queue.cc:260] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 159, Last appended: 1.160, Last appended by leader: 160, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:10.190228 6875 ts_tablet_manager.cc:1434] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.050s user 0.002s sys 0.000s
I20251211 20:55:10.190644 6875 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Bootstrap starting.
I20251211 20:55:10.377580 7035 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:10.378822 7035 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:10.382706 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:10.383535 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 160 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:10.384855 7035 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:10.385244 6872 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:10.385614 7035 raft_consensus.cc:2749] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:10.934197 7013 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:10.938086 6946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.195:38469 (request call id 1) took 2567 ms. Trace:
I20251211 20:55:10.934435 7013 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:10.961616 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:10.962857 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:10.995308 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:10.995810 7035 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:11.001734 7013 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:11.094895 7014 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:11.098088 6943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.194:47609 (request call id 1) took 2716 ms. Trace:
I20251211 20:55:11.095132 7014 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:11.123049 7014 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:11.123773 6940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:42702 (request call id 7) took 2565 ms. Trace:
I20251211 20:55:11.123867 6940 rpcz_store.cc:276] 1211 20:55:08.558571 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:08.558623 (+ 52us) service_pool.cc:225] Handling call
1211 20:55:11.123529 (+2564906us) spinlock_profiling.cc:243] Waited 2.56 s on lock 0x616000019054. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 00007fdce26306bd 00007fdce26307b3 00007fdcdf1f7521 00007fdce306129b 00007fdcdd1f1b07 00007fdcdd1f1835 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685 00007fdcda2cefaf 00007fdcda2cedd8
1211 20:55:11.123721 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5897601920}
I20251211 20:55:11.125315 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:11.128829 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:11.146119 7028 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:11.146380 7028 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:11.147506 7028 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:11.152423 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:11.160939 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 910 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:11.167541 7028 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:11.168462 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 910 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:11.182632 6947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.195:38469 (request call id 3) took 2422 ms. Trace:
I20251211 20:55:11.192685 6872 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:11.194384 7028 raft_consensus.cc:2749] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:11.442101 7028 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:11.442382 7028 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:11.443223 7028 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:11.443656 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:11.443704 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:11.444298 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:11.444689 7028 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:11.468055 6944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.194:47609 (request call id 0) took 3100 ms. Trace:
I20251211 20:55:11.468470 7015 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:11.468622 7015 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:11.469489 7015 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:11.472026 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 878 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:11.486860 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 878 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:11.530115 6872 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:11.530710 7015 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:11.599490 7015 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:11.607177 7028 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:11.624490 7028 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:11.625023 7015 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:11.626341 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 910 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:11.630249 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 910 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:11.637033 7015 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
W20251211 20:55:11.690054 7029 log.cc:927] Time spent T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89: Append to log took a long time: real 0.052s user 0.000s sys 0.000s
I20251211 20:55:11.691095 6872 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:11.691825 7015 raft_consensus.cc:2749] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:11.692023 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:11.692700 7028 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:11.693037 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:11.698782 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:11.732846 7028 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:11.843523 7028 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:11.843552 7015 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:11.844228 7028 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:11.847122 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:11.846264 7015 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:11.850659 7028 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:11.850804 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 878 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:11.853183 7015 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:11.858048 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 878 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:11.859105 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:11.863387 6872 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:11.869374 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:11.869814 7015 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:11.873929 7015 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:12.042165 7015 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:12.044772 7015 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:12.045514 7015 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:12.046208 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:12.046255 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:12.076879 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:12.077998 7015 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:12.339519 7015 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:12.342692 7015 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:12.343396 7015 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:12.345706 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:12.346743 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:12.366967 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:12.367420 7015 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:12.369287 7028 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:12.369484 7015 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:12.371428 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 910 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:12.371585 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 910 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:12.374469 7015 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:12.374600 6872 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:12.380151 7015 raft_consensus.cc:2749] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:12.438058 7015 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:12.438242 7015 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:12.438910 7015 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:12.439911 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:12.442902 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:12.447121 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:12.447407 7015 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:12.496632 7015 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:12.496848 7015 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:12.497494 7015 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:12.498421 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 878 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:12.500039 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 878 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:12.501741 6872 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:12.502187 7015 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:12.868703 7015 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:12.868911 7015 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:12.869560 7015 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:12.870093 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 910 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:12.873176 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 910 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:12.875806 6872 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:12.877270 7015 raft_consensus.cc:2749] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:55:13.059294 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 121: this message will repeat every 5th retry.
I20251211 20:55:11.182848 6947 rpcz_store.cc:276] 1211 20:55:08.721893 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:08.722619 (+ 726us) service_pool.cc:225] Handling call
1211 20:55:08.858103 (+135484us) raft_consensus.cc:1471] Updating replica for 750 ops
1211 20:55:08.953195 (+ 95092us) raft_consensus.cc:1534] Early marking committed up to index 160
1211 20:55:08.959389 (+ 6194us) raft_consensus.cc:1539] Triggering prepare for 750 ops
1211 20:55:08.959537 (+ 148us) write_op.cc:270] Start()
1211 20:55:08.959561 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495510633 usec, L: 0
1211 20:55:09.064603 (+105042us) write_op.cc:270] Start()
1211 20:55:09.064627 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495535134 usec, L: 0
1211 20:55:09.064812 (+ 185us) write_op.cc:270] Start()
1211 20:55:09.064837 (+ 25us) write_op.cc:276] Timestamp: P: 1765486495545682 usec, L: 0
1211 20:55:09.065009 (+ 172us) write_op.cc:270] Start()
1211 20:55:09.065029 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495554641 usec, L: 0
1211 20:55:09.065254 (+ 225us) write_op.cc:270] Start()
1211 20:55:09.065275 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495570265 usec, L: 0
1211 20:55:09.065426 (+ 151us) write_op.cc:270] Start()
1211 20:55:09.065447 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495585680 usec, L: 0
1211 20:55:09.065587 (+ 140us) write_op.cc:270] Start()
1211 20:55:09.065625 (+ 38us) write_op.cc:276] Timestamp: P: 1765486495606344 usec, L: 0
1211 20:55:09.066623 (+ 998us) write_op.cc:270] Start()
1211 20:55:09.066646 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495610342 usec, L: 0
1211 20:55:09.066837 (+ 191us) write_op.cc:270] Start()
1211 20:55:09.066857 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495619718 usec, L: 0
1211 20:55:09.067053 (+ 196us) write_op.cc:270] Start()
1211 20:55:09.068397 (+ 1344us) write_op.cc:276] Timestamp: P: 1765486495651381 usec, L: 0
1211 20:55:09.068573 (+ 176us) write_op.cc:270] Start()
1211 20:55:09.068592 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495652732 usec, L: 0
1211 20:55:09.068834 (+ 242us) write_op.cc:270] Start()
1211 20:55:09.068856 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495693135 usec, L: 0
1211 20:55:09.069026 (+ 170us) write_op.cc:270] Start()
1211 20:55:09.069045 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495703047 usec, L: 0
1211 20:55:09.069201 (+ 156us) write_op.cc:270] Start()
1211 20:55:09.069220 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495713001 usec, L: 0
1211 20:55:09.069366 (+ 146us) write_op.cc:270] Start()
1211 20:55:09.069386 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495723579 usec, L: 0
1211 20:55:09.069521 (+ 135us) write_op.cc:270] Start()
1211 20:55:09.069538 (+ 17us) write_op.cc:276] Timestamp: P: 1765486495726450 usec, L: 0
1211 20:55:09.069711 (+ 173us) write_op.cc:270] Start()
1211 20:55:09.069757 (+ 46us) write_op.cc:276] Timestamp: P: 1765486495745373 usec, L: 0
1211 20:55:09.069906 (+ 149us) write_op.cc:270] Start()
1211 20:55:09.069925 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495767068 usec, L: 0
1211 20:55:09.070096 (+ 171us) write_op.cc:270] Start()
1211 20:55:09.070126 (+ 30us) write_op.cc:276] Timestamp: P: 1765486495773947 usec, L: 0
1211 20:55:09.070275 (+ 149us) write_op.cc:270] Start()
1211 20:55:09.070295 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495798743 usec, L: 0
1211 20:55:09.070450 (+ 155us) write_op.cc:270] Start()
1211 20:55:09.070469 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495801329 usec, L: 0
1211 20:55:09.070594 (+ 125us) write_op.cc:270] Start()
1211 20:55:09.070611 (+ 17us) write_op.cc:276] Timestamp: P: 1765486495820477 usec, L: 0
1211 20:55:09.070769 (+ 158us) write_op.cc:270] Start()
1211 20:55:09.070791 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495864974 usec, L: 0
1211 20:55:09.070961 (+ 170us) write_op.cc:270] Start()
1211 20:55:09.070983 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495881874 usec, L: 0
1211 20:55:09.071230 (+ 247us) write_op.cc:270] Start()
1211 20:55:09.071258 (+ 28us) write_op.cc:276] Timestamp: P: 1765486495893490 usec, L: 0
1211 20:55:09.071490 (+ 232us) write_op.cc:270] Start()
1211 20:55:09.071514 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495898896 usec, L: 0
1211 20:55:09.071700 (+ 186us) write_op.cc:270] Start()
1211 20:55:09.071721 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495911563 usec, L: 0
1211 20:55:09.071888 (+ 167us) write_op.cc:270] Start()
1211 20:55:09.071908 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495936319 usec, L: 0
1211 20:55:09.072059 (+ 151us) write_op.cc:270] Start()
1211 20:55:09.072081 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495949004 usec, L: 0
1211 20:55:09.072269 (+ 188us) write_op.cc:270] Start()
1211 20:55:09.072291 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495959065 usec, L: 0
1211 20:55:09.072490 (+ 199us) write_op.cc:270] Start()
1211 20:55:09.072513 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495960870 usec, L: 0
1211 20:55:09.072669 (+ 156us) write_op.cc:270] Start()
1211 20:55:09.072691 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495967262 usec, L: 0
1211 20:55:09.072931 (+ 240us) write_op.cc:270] Start()
1211 20:55:09.072973 (+ 42us) write_op.cc:276] Timestamp: P: 1765486496002317 usec, L: 0
1211 20:55:09.073171 (+ 198us) write_op.cc:270] Start()
1211 20:55:09.073193 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496008095 usec, L: 0
1211 20:55:09.073381 (+ 188us) write_op.cc:270] Start()
1211 20:55:09.073418 (+ 37us) write_op.cc:276] Timestamp: P: 1765486496021625 usec, L: 0
1211 20:55:09.073570 (+ 152us) write_op.cc:270] Start()
1211 20:55:09.073592 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496032297 usec, L: 0
1211 20:55:09.073750 (+ 158us) write_op.cc:270] Start()
1211 20:55:09.073783 (+ 33us) write_op.cc:276] Timestamp: P: 1765486496057413 usec, L: 0
1211 20:55:09.073931 (+ 148us) write_op.cc:270] Start()
1211 20:55:09.073960 (+ 29us) write_op.cc:276] Timestamp: P: 1765486496060614 usec, L: 0
1211 20:55:09.074147 (+ 187us) write_op.cc:270] Start()
1211 20:55:09.074166 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496073783 usec, L: 0
1211 20:55:09.074307 (+ 141us) write_op.cc:270] Start()
1211 20:55:09.074325 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496083559 usec, L: 0
1211 20:55:09.074449 (+ 124us) write_op.cc:270] Start()
1211 20:55:09.074470 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496101335 usec, L: 0
1211 20:55:09.074675 (+ 205us) write_op.cc:270] Start()
1211 20:55:09.074695 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496118112 usec, L: 0
1211 20:55:09.074841 (+ 146us) write_op.cc:270] Start()
1211 20:55:09.074861 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496123637 usec, L: 0
1211 20:55:09.075005 (+ 144us) write_op.cc:270] Start()
1211 20:55:09.075024 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496141223 usec, L: 0
1211 20:55:09.075172 (+ 148us) write_op.cc:270] Start()
1211 20:55:09.075191 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496143283 usec, L: 0
1211 20:55:09.075388 (+ 197us) write_op.cc:270] Start()
1211 20:55:09.075409 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496153348 usec, L: 0
1211 20:55:09.081699 (+ 6290us) write_op.cc:270] Start()
1211 20:55:09.081726 (+ 27us) write_op.cc:276] Timestamp: P: 1765486496154949 usec, L: 0
1211 20:55:09.081896 (+ 170us) write_op.cc:270] Start()
1211 20:55:09.081919 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496186191 usec, L: 0
1211 20:55:09.082091 (+ 172us) write_op.cc:270] Start()
1211 20:55:09.082124 (+ 33us) write_op.cc:276] Timestamp: P: 1765486496189013 usec, L: 0
1211 20:55:09.082326 (+ 202us) write_op.cc:270] Start()
1211 20:55:09.082348 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496200621 usec, L: 0
1211 20:55:09.082540 (+ 192us) write_op.cc:270] Start()
1211 20:55:09.082565 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496219723 usec, L: 0
1211 20:55:09.082721 (+ 156us) write_op.cc:270] Start()
1211 20:55:09.082741 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496239294 usec, L: 0
1211 20:55:09.082894 (+ 153us) write_op.cc:270] Start()
1211 20:55:09.082914 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496248016 usec, L: 0
1211 20:55:09.083050 (+ 136us) write_op.cc:270] Start()
1211 20:55:09.083073 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496263286 usec, L: 0
1211 20:55:09.083238 (+ 165us) write_op.cc:270] Start()
1211 20:55:09.083262 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496278627 usec, L: 0
1211 20:55:09.083482 (+ 220us) write_op.cc:270] Start()
1211 20:55:09.083505 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496284168 usec, L: 0
1211 20:55:09.083674 (+ 169us) write_op.cc:270] Start()
1211 20:55:09.083695 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496294642 usec, L: 0
1211 20:55:09.083829 (+ 134us) write_op.cc:270] Start()
1211 20:55:09.083849 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496316355 usec, L: 0
1211 20:55:09.084012 (+ 163us) write_op.cc:270] Start()
1211 20:55:09.084031 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496335265 usec, L: 0
1211 20:55:09.084185 (+ 154us) write_op.cc:270] Start()
1211 20:55:09.084221 (+ 36us) write_op.cc:276] Timestamp: P: 1765486496337896 usec, L: 0
1211 20:55:09.084362 (+ 141us) write_op.cc:270] Start()
1211 20:55:09.084384 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496359287 usec, L: 0
1211 20:55:09.084548 (+ 164us) write_op.cc:270] Start()
1211 20:55:09.084571 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496376824 usec, L: 0
1211 20:55:09.084726 (+ 155us) write_op.cc:270] Start()
1211 20:55:09.084749 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496392868 usec, L: 0
1211 20:55:09.084900 (+ 151us) write_op.cc:270] Start()
1211 20:55:09.084923 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496403541 usec, L: 0
1211 20:55:09.085178 (+ 255us) write_op.cc:270] Start()
1211 20:55:09.085205 (+ 27us) write_op.cc:276] Timestamp: P: 1765486496410392 usec, L: 0
1211 20:55:09.085382 (+ 177us) write_op.cc:270] Start()
1211 20:55:09.085403 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496419501 usec, L: 0
1211 20:55:09.085606 (+ 203us) write_op.cc:270] Start()
1211 20:55:09.085626 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496436995 usec, L: 0
1211 20:55:09.099906 (+ 14280us) write_op.cc:270] Start()
1211 20:55:09.099929 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496449471 usec, L: 0
1211 20:55:09.100099 (+ 170us) write_op.cc:270] Start()
1211 20:55:09.100130 (+ 31us) write_op.cc:276] Timestamp: P: 1765486496469602 usec, L: 0
1211 20:55:09.100291 (+ 161us) write_op.cc:270] Start()
1211 20:55:09.100316 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496488141 usec, L: 0
1211 20:55:09.100470 (+ 154us) write_op.cc:270] Start()
1211 20:55:09.100490 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496489076 usec, L: 0
1211 20:55:09.104331 (+ 3841us) write_op.cc:270] Start()
1211 20:55:09.104351 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496491970 usec, L: 0
1211 20:55:09.104559 (+ 208us) write_op.cc:270] Start()
1211 20:55:09.104578 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496505878 usec, L: 0
1211 20:55:09.104705 (+ 127us) write_op.cc:270] Start()
1211 20:55:09.104723 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496523123 usec, L: 0
1211 20:55:09.114055 (+ 9332us) write_op.cc:270] Start()
1211 20:55:09.114079 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496528808 usec, L: 0
1211 20:55:09.114257 (+ 178us) write_op.cc:270] Start()
1211 20:55:09.114279 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496536166 usec, L: 0
1211 20:55:09.114514 (+ 235us) write_op.cc:270] Start()
1211 20:55:09.114535 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496556741 usec, L: 0
1211 20:55:09.114712 (+ 177us) write_op.cc:270] Start()
1211 20:55:09.114733 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496571039 usec, L: 0
1211 20:55:09.114897 (+ 164us) write_op.cc:270] Start()
1211 20:55:09.114952 (+ 55us) write_op.cc:276] Timestamp: P: 1765486496573825 usec, L: 0
1211 20:55:09.115109 (+ 157us) write_op.cc:270] Start()
1211 20:55:09.115126 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496580886 usec, L: 0
1211 20:55:09.115286 (+ 160us) write_op.cc:270] Start()
1211 20:55:09.115306 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496608919 usec, L: 0
1211 20:55:09.115471 (+ 165us) write_op.cc:270] Start()
1211 20:55:09.115491 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496624079 usec, L: 0
1211 20:55:09.115622 (+ 131us) write_op.cc:270] Start()
1211 20:55:09.115639 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496626285 usec, L: 0
1211 20:55:09.115759 (+ 120us) write_op.cc:270] Start()
1211 20:55:09.115779 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496642745 usec, L: 0
1211 20:55:09.115915 (+ 136us) write_op.cc:270] Start()
1211 20:55:09.115944 (+ 29us) write_op.cc:276] Timestamp: P: 1765486496648031 usec, L: 0
1211 20:55:09.116067 (+ 123us) write_op.cc:270] Start()
1211 20:55:09.116107 (+ 40us) write_op.cc:276] Timestamp: P: 1765486496662026 usec, L: 0
1211 20:55:09.116284 (+ 177us) write_op.cc:270] Start()
1211 20:55:09.116303 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496693416 usec, L: 0
1211 20:55:09.116521 (+ 218us) write_op.cc:270] Start()
1211 20:55:09.116540 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496694062 usec, L: 0
1211 20:55:09.116680 (+ 140us) write_op.cc:270] Start()
1211 20:55:09.116697 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496719405 usec, L: 0
1211 20:55:09.116848 (+ 151us) write_op.cc:270] Start()
1211 20:55:09.116867 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496720318 usec, L: 0
1211 20:55:09.116996 (+ 129us) write_op.cc:270] Start()
1211 20:55:09.117014 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496743772 usec, L: 0
1211 20:55:09.117174 (+ 160us) write_op.cc:270] Start()
1211 20:55:09.117195 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496750150 usec, L: 0
1211 20:55:09.117335 (+ 140us) write_op.cc:270] Start()
1211 20:55:09.117355 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496758252 usec, L: 0
1211 20:55:09.117526 (+ 171us) write_op.cc:270] Start()
1211 20:55:09.117542 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496787157 usec, L: 0
1211 20:55:09.117675 (+ 133us) write_op.cc:270] Start()
1211 20:55:09.117693 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496789099 usec, L: 0
1211 20:55:09.117862 (+ 169us) write_op.cc:270] Start()
1211 20:55:09.117881 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496812604 usec, L: 0
1211 20:55:09.154058 (+ 36177us) write_op.cc:270] Start()
1211 20:55:09.154077 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496816520 usec, L: 0
1211 20:55:09.154246 (+ 169us) write_op.cc:270] Start()
1211 20:55:09.154262 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496819921 usec, L: 0
1211 20:55:09.154406 (+ 144us) write_op.cc:270] Start()
1211 20:55:09.154430 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496854640 usec, L: 0
1211 20:55:09.154591 (+ 161us) write_op.cc:270] Start()
1211 20:55:09.154610 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496866450 usec, L: 0
1211 20:55:09.154743 (+ 133us) write_op.cc:270] Start()
1211 20:55:09.154761 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496875065 usec, L: 0
1211 20:55:09.154963 (+ 202us) write_op.cc:270] Start()
1211 20:55:09.154985 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496882581 usec, L: 0
1211 20:55:09.155125 (+ 140us) write_op.cc:270] Start()
1211 20:55:09.155143 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496915274 usec, L: 0
1211 20:55:09.155298 (+ 155us) write_op.cc:270] Start()
1211 20:55:09.155317 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496931360 usec, L: 0
1211 20:55:09.155511 (+ 194us) write_op.cc:270] Start()
1211 20:55:09.155531 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496940008 usec, L: 0
1211 20:55:09.155672 (+ 141us) write_op.cc:270] Start()
1211 20:55:09.155690 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496958820 usec, L: 0
1211 20:55:09.155846 (+ 156us) write_op.cc:270] Start()
1211 20:55:09.155867 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496970203 usec, L: 0
1211 20:55:09.156097 (+ 230us) write_op.cc:270] Start()
1211 20:55:09.156118 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496995385 usec, L: 0
1211 20:55:09.156291 (+ 173us) write_op.cc:270] Start()
1211 20:55:09.156311 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496997130 usec, L: 0
1211 20:55:09.156481 (+ 170us) write_op.cc:270] Start()
1211 20:55:09.156503 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496997784 usec, L: 0
1211 20:55:09.156655 (+ 152us) write_op.cc:270] Start()
1211 20:55:09.156674 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497016227 usec, L: 0
1211 20:55:09.156820 (+ 146us) write_op.cc:270] Start()
1211 20:55:09.156839 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497039420 usec, L: 0
1211 20:55:09.156980 (+ 141us) write_op.cc:270] Start()
1211 20:55:09.157001 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497062114 usec, L: 0
1211 20:55:09.157166 (+ 165us) write_op.cc:270] Start()
1211 20:55:09.157188 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497075344 usec, L: 0
1211 20:55:09.157342 (+ 154us) write_op.cc:270] Start()
1211 20:55:09.157368 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497081522 usec, L: 0
1211 20:55:09.165604 (+ 8236us) write_op.cc:270] Start()
1211 20:55:09.165627 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497083846 usec, L: 0
1211 20:55:09.165813 (+ 186us) write_op.cc:270] Start()
1211 20:55:09.165837 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497089594 usec, L: 0
1211 20:55:09.173349 (+ 7512us) write_op.cc:270] Start()
1211 20:55:09.173379 (+ 30us) write_op.cc:276] Timestamp: P: 1765486497119344 usec, L: 0
1211 20:55:09.188576 (+ 15197us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x60f0000180c8. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fdcdc2b8f78 00007fdcdf230dc9 00007fdcdf1d7eb9 00007fdcdf1d56c7 00007fdce304f21d 00007fdcdd1e8b46 00007fdcdd1e8875 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685
1211 20:55:09.188695 (+ 119us) write_op.cc:270] Start()
1211 20:55:09.188723 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497131918 usec, L: 0
1211 20:55:09.188911 (+ 188us) write_op.cc:270] Start()
1211 20:55:09.188931 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497135166 usec, L: 0
1211 20:55:09.189067 (+ 136us) write_op.cc:270] Start()
1211 20:55:09.189085 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497143096 usec, L: 0
1211 20:55:09.189242 (+ 157us) write_op.cc:270] Start()
1211 20:55:09.189263 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497165864 usec, L: 0
1211 20:55:09.189431 (+ 168us) write_op.cc:270] Start()
1211 20:55:09.189451 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497180405 usec, L: 0
1211 20:55:09.189635 (+ 184us) write_op.cc:270] Start()
1211 20:55:09.189654 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497181064 usec, L: 0
1211 20:55:09.189784 (+ 130us) write_op.cc:270] Start()
1211 20:55:09.189804 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497221387 usec, L: 0
1211 20:55:09.189935 (+ 131us) write_op.cc:270] Start()
1211 20:55:09.189953 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497222164 usec, L: 0
1211 20:55:09.190172 (+ 219us) write_op.cc:270] Start()
1211 20:55:09.190194 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497229122 usec, L: 0
1211 20:55:09.190402 (+ 208us) write_op.cc:270] Start()
1211 20:55:09.190452 (+ 50us) write_op.cc:276] Timestamp: P: 1765486497233593 usec, L: 0
1211 20:55:09.191952 (+ 1500us) write_op.cc:270] Start()
1211 20:55:09.191974 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497260066 usec, L: 0
1211 20:55:09.192164 (+ 190us) write_op.cc:270] Start()
1211 20:55:09.192185 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497293088 usec, L: 0
1211 20:55:09.192383 (+ 198us) write_op.cc:270] Start()
1211 20:55:09.192405 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497294600 usec, L: 0
1211 20:55:09.192553 (+ 148us) write_op.cc:270] Start()
1211 20:55:09.192574 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497301974 usec, L: 0
1211 20:55:09.192725 (+ 151us) write_op.cc:270] Start()
1211 20:55:09.192746 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497326192 usec, L: 0
1211 20:55:09.192913 (+ 167us) write_op.cc:270] Start()
1211 20:55:09.192933 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497349290 usec, L: 0
1211 20:55:09.204386 (+ 11453us) write_op.cc:270] Start()
1211 20:55:09.204413 (+ 27us) write_op.cc:276] Timestamp: P: 1765486497352150 usec, L: 0
1211 20:55:09.238558 (+ 34145us) write_op.cc:270] Start()
1211 20:55:09.238581 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497354698 usec, L: 0
1211 20:55:09.238744 (+ 163us) write_op.cc:270] Start()
1211 20:55:09.238763 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497378652 usec, L: 0
1211 20:55:09.239575 (+ 812us) write_op.cc:270] Start()
1211 20:55:09.239603 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497399885 usec, L: 0
1211 20:55:09.247626 (+ 8023us) write_op.cc:270] Start()
1211 20:55:09.247655 (+ 29us) write_op.cc:276] Timestamp: P: 1765486497409734 usec, L: 0
1211 20:55:09.254821 (+ 7166us) write_op.cc:270] Start()
1211 20:55:09.254843 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497416875 usec, L: 0
1211 20:55:09.279625 (+ 24782us) write_op.cc:270] Start()
1211 20:55:09.279648 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497439268 usec, L: 0
1211 20:55:09.281440 (+ 1792us) write_op.cc:270] Start()
1211 20:55:09.281464 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497440110 usec, L: 0
1211 20:55:09.281625 (+ 161us) write_op.cc:270] Start()
1211 20:55:09.281646 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497444996 usec, L: 0
1211 20:55:09.284781 (+ 3135us) write_op.cc:270] Start()
1211 20:55:09.284801 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497458398 usec, L: 0
1211 20:55:09.284948 (+ 147us) write_op.cc:270] Start()
1211 20:55:09.284964 (+ 16us) write_op.cc:276] Timestamp: P: 1765486497487980 usec, L: 0
1211 20:55:09.285779 (+ 815us) write_op.cc:270] Start()
1211 20:55:09.285800 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497493079 usec, L: 0
1211 20:55:09.289330 (+ 3530us) write_op.cc:270] Start()
1211 20:55:09.289348 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497494126 usec, L: 0
1211 20:55:09.289465 (+ 117us) write_op.cc:270] Start()
1211 20:55:09.289482 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497507988 usec, L: 0
1211 20:55:09.290166 (+ 684us) write_op.cc:270] Start()
1211 20:55:09.290192 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497539438 usec, L: 0
1211 20:55:09.296568 (+ 6376us) write_op.cc:270] Start()
1211 20:55:09.296595 (+ 27us) write_op.cc:276] Timestamp: P: 1765486497544455 usec, L: 0
1211 20:55:09.297741 (+ 1146us) write_op.cc:270] Start()
1211 20:55:09.297764 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497555953 usec, L: 0
1211 20:55:09.300237 (+ 2473us) write_op.cc:270] Start()
1211 20:55:09.300262 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497556727 usec, L: 0
1211 20:55:09.300425 (+ 163us) write_op.cc:270] Start()
1211 20:55:09.300446 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497578982 usec, L: 0
1211 20:55:09.300577 (+ 131us) write_op.cc:270] Start()
1211 20:55:09.300595 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497599071 usec, L: 0
1211 20:55:09.303966 (+ 3371us) write_op.cc:270] Start()
1211 20:55:09.303986 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497610185 usec, L: 0
1211 20:55:09.307091 (+ 3105us) write_op.cc:270] Start()
1211 20:55:09.307117 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497622969 usec, L: 0
1211 20:55:09.307300 (+ 183us) write_op.cc:270] Start()
1211 20:55:09.307323 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497645706 usec, L: 0
1211 20:55:09.312577 (+ 5254us) write_op.cc:270] Start()
1211 20:55:09.312602 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497657422 usec, L: 0
1211 20:55:09.321599 (+ 8997us) write_op.cc:270] Start()
1211 20:55:09.321613 (+ 14us) write_op.cc:276] Timestamp: P: 1765486497663230 usec, L: 0
1211 20:55:09.321736 (+ 123us) write_op.cc:270] Start()
1211 20:55:09.321751 (+ 15us) write_op.cc:276] Timestamp: P: 1765486497682566 usec, L: 0
1211 20:55:09.322287 (+ 536us) write_op.cc:270] Start()
1211 20:55:09.322309 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497687672 usec, L: 0
1211 20:55:09.322444 (+ 135us) write_op.cc:270] Start()
1211 20:55:09.322461 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497706301 usec, L: 0
1211 20:55:09.322587 (+ 126us) write_op.cc:270] Start()
1211 20:55:09.322603 (+ 16us) write_op.cc:276] Timestamp: P: 1765486497723602 usec, L: 0
1211 20:55:09.322739 (+ 136us) write_op.cc:270] Start()
1211 20:55:09.322757 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497730431 usec, L: 0
1211 20:55:09.322876 (+ 119us) write_op.cc:270] Start()
1211 20:55:09.322894 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497748937 usec, L: 0
1211 20:55:09.323012 (+ 118us) write_op.cc:270] Start()
1211 20:55:09.323029 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497771635 usec, L: 0
1211 20:55:09.323500 (+ 471us) write_op.cc:270] Start()
1211 20:55:09.343304 (+ 19804us) write_op.cc:276] Timestamp: P: 1765486497775399 usec, L: 0
1211 20:55:09.344249 (+ 945us) write_op.cc:270] Start()
1211 20:55:09.344266 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497781923 usec, L: 0
1211 20:55:09.347679 (+ 3413us) write_op.cc:270] Start()
1211 20:55:09.347702 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497797630 usec, L: 0
1211 20:55:09.349772 (+ 2070us) write_op.cc:270] Start()
1211 20:55:09.349792 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497813501 usec, L: 0
1211 20:55:09.349921 (+ 129us) write_op.cc:270] Start()
1211 20:55:09.349940 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497818517 usec, L: 0
1211 20:55:09.359272 (+ 9332us) write_op.cc:270] Start()
1211 20:55:09.359293 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497843796 usec, L: 0
1211 20:55:09.364995 (+ 5702us) write_op.cc:270] Start()
1211 20:55:09.365009 (+ 14us) write_op.cc:276] Timestamp: P: 1765486497850181 usec, L: 0
1211 20:55:09.371986 (+ 6977us) write_op.cc:270] Start()
1211 20:55:09.372011 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497864193 usec, L: 0
1211 20:55:09.372195 (+ 184us) write_op.cc:270] Start()
1211 20:55:09.372218 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497882048 usec, L: 0
1211 20:55:09.379876 (+ 7658us) write_op.cc:270] Start()
1211 20:55:09.379899 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497887947 usec, L: 0
1211 20:55:09.382397 (+ 2498us) write_op.cc:270] Start()
1211 20:55:09.382421 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497899428 usec, L: 0
1211 20:55:09.389809 (+ 7388us) write_op.cc:270] Start()
1211 20:55:09.389835 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497929958 usec, L: 0
1211 20:55:09.392534 (+ 2699us) write_op.cc:270] Start()
1211 20:55:09.392559 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497941037 usec, L: 0
1211 20:55:09.398385 (+ 5826us) write_op.cc:270] Start()
1211 20:55:09.398411 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497946925 usec, L: 0
1211 20:55:09.407236 (+ 8825us) write_op.cc:270] Start()
1211 20:55:09.407257 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497955213 usec, L: 0
1211 20:55:09.407433 (+ 176us) write_op.cc:270] Start()
1211 20:55:09.407455 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497985144 usec, L: 0
1211 20:55:09.407636 (+ 181us) write_op.cc:270] Start()
1211 20:55:09.407658 (+ 22us) write_op.cc:276] Timestamp: P: 1765486498010720 usec, L: 0
1211 20:55:09.407821 (+ 163us) write_op.cc:270] Start()
1211 20:55:09.407844 (+ 23us) write_op.cc:276] Timestamp: P: 1765486498031680 usec, L: 0
1211 20:55:09.407996 (+ 152us) write_op.cc:270] Start()
1211 20:55:09.408016 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498035421 usec, L: 0
1211 20:55:09.408147 (+ 131us) write_op.cc:270] Start()
1211 20:55:09.408166 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498038962 usec, L: 0
1211 20:55:09.408317 (+ 151us) write_op.cc:270] Start()
1211 20:55:09.408337 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498048970 usec, L: 0
1211 20:55:09.408476 (+ 139us) write_op.cc:270] Start()
1211 20:55:09.408495 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498080313 usec, L: 0
1211 20:55:09.408641 (+ 146us) write_op.cc:270] Start()
1211 20:55:09.408660 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498089534 usec, L: 0
1211 20:55:09.408799 (+ 139us) write_op.cc:270] Start()
1211 20:55:09.408819 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498105505 usec, L: 0
1211 20:55:09.408948 (+ 129us) write_op.cc:270] Start()
1211 20:55:09.408967 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498128981 usec, L: 0
1211 20:55:09.409113 (+ 146us) write_op.cc:270] Start()
1211 20:55:09.409134 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498131376 usec, L: 0
1211 20:55:09.410645 (+ 1511us) write_op.cc:270] Start()
1211 20:55:09.410670 (+ 25us) write_op.cc:276] Timestamp: P: 1765486498135799 usec, L: 0
1211 20:55:09.427188 (+ 16518us) write_op.cc:270] Start()
1211 20:55:09.427215 (+ 27us) write_op.cc:276] Timestamp: P: 1765486498154910 usec, L: 0
1211 20:55:09.464962 (+ 37747us) write_op.cc:270]
I20251211 20:55:13.375691 7015 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:13.375892 7015 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:13.379634 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 878 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:13.386579 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 878 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:13.397830 6872 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:13.401077 7015 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:13.422024 7015 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:13.526470 7015 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:13.526683 7015 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:13.528620 7015 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
W20251211 20:55:13.583578 6872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.211.195:46117, user_credentials={real_user=slave}} blocked reactor thread for 136440us
I20251211 20:55:13.589915 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:13.591679 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:13.650599 7051 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:13.626170 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:13.660965 7015 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:13.650893 7051 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:13.676656 7051 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:11.468292 6944 rpcz_store.cc:276] 1211 20:55:08.367769 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:08.373098 (+ 5329us) service_pool.cc:225] Handling call
1211 20:55:08.567083 (+193985us) raft_consensus.cc:1471] Updating replica for 718 ops
1211 20:55:08.711589 (+144506us) raft_consensus.cc:1534] Early marking committed up to index 160
1211 20:55:08.762329 (+ 50740us) raft_consensus.cc:1539] Triggering prepare for 718 ops
1211 20:55:08.762522 (+ 193us) write_op.cc:270] Start()
1211 20:55:08.762555 (+ 33us) write_op.cc:276] Timestamp: P: 1765486495506217 usec, L: 0
1211 20:55:08.763532 (+ 977us) write_op.cc:270] Start()
1211 20:55:08.763568 (+ 36us) write_op.cc:276] Timestamp: P: 1765486495531496 usec, L: 0
1211 20:55:08.763753 (+ 185us) write_op.cc:270] Start()
1211 20:55:08.763777 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495543358 usec, L: 0
1211 20:55:08.763956 (+ 179us) write_op.cc:270] Start()
1211 20:55:08.763979 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495556869 usec, L: 0
1211 20:55:08.764159 (+ 180us) write_op.cc:270] Start()
1211 20:55:08.764183 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495567677 usec, L: 0
1211 20:55:08.764340 (+ 157us) write_op.cc:270] Start()
1211 20:55:08.764369 (+ 29us) write_op.cc:276] Timestamp: P: 1765486495583895 usec, L: 0
1211 20:55:08.773789 (+ 9420us) write_op.cc:270] Start()
1211 20:55:08.773831 (+ 42us) write_op.cc:276] Timestamp: P: 1765486495610363 usec, L: 0
1211 20:55:08.794872 (+ 21041us) write_op.cc:270] Start()
1211 20:55:08.794894 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495611554 usec, L: 0
1211 20:55:08.795276 (+ 382us) write_op.cc:270] Start()
1211 20:55:08.795299 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495624983 usec, L: 0
1211 20:55:08.795455 (+ 156us) write_op.cc:270] Start()
1211 20:55:08.795473 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495650031 usec, L: 0
1211 20:55:08.795616 (+ 143us) write_op.cc:270] Start()
1211 20:55:08.795642 (+ 26us) write_op.cc:276] Timestamp: P: 1765486495658718 usec, L: 0
1211 20:55:08.795799 (+ 157us) write_op.cc:270] Start()
1211 20:55:08.795830 (+ 31us) write_op.cc:276] Timestamp: P: 1765486495679110 usec, L: 0
1211 20:55:08.795967 (+ 137us) write_op.cc:270] Start()
1211 20:55:08.795984 (+ 17us) write_op.cc:276] Timestamp: P: 1765486495692195 usec, L: 0
1211 20:55:08.796113 (+ 129us) write_op.cc:270] Start()
1211 20:55:08.796132 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495692746 usec, L: 0
1211 20:55:08.796261 (+ 129us) write_op.cc:270] Start()
1211 20:55:08.796277 (+ 16us) write_op.cc:276] Timestamp: P: 1765486495718929 usec, L: 0
1211 20:55:08.796409 (+ 132us) write_op.cc:270] Start()
1211 20:55:08.796447 (+ 38us) write_op.cc:276] Timestamp: P: 1765486495726938 usec, L: 0
1211 20:55:08.796608 (+ 161us) write_op.cc:270] Start()
1211 20:55:08.796627 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495747689 usec, L: 0
1211 20:55:08.797095 (+ 468us) write_op.cc:270] Start()
1211 20:55:08.797118 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495762058 usec, L: 0
1211 20:55:08.797269 (+ 151us) write_op.cc:270] Start()
1211 20:55:08.797288 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495772518 usec, L: 0
1211 20:55:08.797456 (+ 168us) write_op.cc:270] Start()
1211 20:55:08.797482 (+ 26us) write_op.cc:276] Timestamp: P: 1765486495800578 usec, L: 0
1211 20:55:08.797627 (+ 145us) write_op.cc:270] Start()
1211 20:55:08.797651 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495803822 usec, L: 0
1211 20:55:08.797802 (+ 151us) write_op.cc:270] Start()
1211 20:55:08.797825 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495836328 usec, L: 0
1211 20:55:08.797966 (+ 141us) write_op.cc:270] Start()
1211 20:55:08.797984 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495859322 usec, L: 0
1211 20:55:08.798194 (+ 210us) write_op.cc:270] Start()
1211 20:55:08.798219 (+ 25us) write_op.cc:276] Timestamp: P: 1765486495881383 usec, L: 0
1211 20:55:08.802132 (+ 3913us) write_op.cc:270] Start()
1211 20:55:08.802175 (+ 43us) write_op.cc:276] Timestamp: P: 1765486495888862 usec, L: 0
1211 20:55:08.802713 (+ 538us) write_op.cc:270] Start()
1211 20:55:08.802735 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495897506 usec, L: 0
1211 20:55:08.802906 (+ 171us) write_op.cc:270] Start()
1211 20:55:08.802926 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495910721 usec, L: 0
1211 20:55:08.803072 (+ 146us) write_op.cc:270] Start()
1211 20:55:08.803109 (+ 37us) write_op.cc:276] Timestamp: P: 1765486495932072 usec, L: 0
1211 20:55:08.803259 (+ 150us) write_op.cc:270] Start()
1211 20:55:08.803286 (+ 27us) write_op.cc:276] Timestamp: P: 1765486495954598 usec, L: 0
1211 20:55:08.804238 (+ 952us) write_op.cc:270] Start()
1211 20:55:08.804261 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495960762 usec, L: 0
1211 20:55:08.804527 (+ 266us) write_op.cc:270] Start()
1211 20:55:08.804547 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495966372 usec, L: 0
1211 20:55:08.806097 (+ 1550us) write_op.cc:270] Start()
1211 20:55:08.806115 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495969353 usec, L: 0
1211 20:55:08.807247 (+ 1132us) write_op.cc:270] Start()
1211 20:55:08.807292 (+ 45us) write_op.cc:276] Timestamp: P: 1765486496004532 usec, L: 0
1211 20:55:08.823150 (+ 15858us) write_op.cc:270] Start()
1211 20:55:08.823164 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496005653 usec, L: 0
1211 20:55:08.824381 (+ 1217us) write_op.cc:270] Start()
1211 20:55:08.824397 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496021279 usec, L: 0
1211 20:55:08.824513 (+ 116us) write_op.cc:270] Start()
1211 20:55:08.824541 (+ 28us) write_op.cc:276] Timestamp: P: 1765486496031820 usec, L: 0
1211 20:55:08.827063 (+ 2522us) write_op.cc:270] Start()
1211 20:55:08.827088 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496058517 usec, L: 0
1211 20:55:08.828216 (+ 1128us) write_op.cc:270] Start()
1211 20:55:08.828240 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496059272 usec, L: 0
1211 20:55:08.828380 (+ 140us) write_op.cc:270] Start()
1211 20:55:08.828398 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496070035 usec, L: 0
1211 20:55:08.829021 (+ 623us) write_op.cc:270] Start()
1211 20:55:08.829042 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496083987 usec, L: 0
1211 20:55:08.829455 (+ 413us) write_op.cc:270] Start()
1211 20:55:08.829474 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496104008 usec, L: 0
1211 20:55:08.830776 (+ 1302us) write_op.cc:270] Start()
1211 20:55:08.830793 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496116115 usec, L: 0
1211 20:55:08.831450 (+ 657us) write_op.cc:270] Start()
1211 20:55:08.831468 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496123382 usec, L: 0
1211 20:55:08.831590 (+ 122us) write_op.cc:270] Start()
1211 20:55:08.831641 (+ 51us) write_op.cc:276] Timestamp: P: 1765486496144593 usec, L: 0
1211 20:55:08.831774 (+ 133us) write_op.cc:270] Start()
1211 20:55:08.831790 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496146792 usec, L: 0
1211 20:55:08.831903 (+ 113us) write_op.cc:270] Start()
1211 20:55:08.831918 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496157691 usec, L: 0
1211 20:55:08.832018 (+ 100us) write_op.cc:270] Start()
1211 20:55:08.832033 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496160767 usec, L: 0
1211 20:55:08.832159 (+ 126us) write_op.cc:270] Start()
1211 20:55:08.832176 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496192573 usec, L: 0
1211 20:55:08.832336 (+ 160us) write_op.cc:270] Start()
1211 20:55:08.832352 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496198354 usec, L: 0
1211 20:55:08.832464 (+ 112us) write_op.cc:270] Start()
1211 20:55:08.832479 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496212131 usec, L: 0
1211 20:55:08.834767 (+ 2288us) write_op.cc:270] Start()
1211 20:55:08.834791 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496213318 usec, L: 0
1211 20:55:08.840312 (+ 5521us) write_op.cc:270] Start()
1211 20:55:08.840332 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496236505 usec, L: 0
1211 20:55:08.840462 (+ 130us) write_op.cc:270] Start()
1211 20:55:08.840479 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496247073 usec, L: 0
1211 20:55:08.841593 (+ 1114us) write_op.cc:270] Start()
1211 20:55:08.841608 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496252947 usec, L: 0
1211 20:55:08.841711 (+ 103us) write_op.cc:270] Start()
1211 20:55:08.841725 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496287008 usec, L: 0
1211 20:55:08.841818 (+ 93us) write_op.cc:270] Start()
1211 20:55:08.841830 (+ 12us) write_op.cc:276] Timestamp: P: 1765486496293031 usec, L: 0
1211 20:55:08.841922 (+ 92us) write_op.cc:270] Start()
1211 20:55:08.841935 (+ 13us) write_op.cc:276] Timestamp: P: 1765486496297339 usec, L: 0
1211 20:55:08.843901 (+ 1966us) write_op.cc:270] Start()
1211 20:55:08.843921 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496321785 usec, L: 0
1211 20:55:08.844047 (+ 126us) write_op.cc:270] Start()
1211 20:55:08.844062 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496335766 usec, L: 0
1211 20:55:08.844177 (+ 115us) write_op.cc:270] Start()
1211 20:55:08.844216 (+ 39us) write_op.cc:276] Timestamp: P: 1765486496340540 usec, L: 0
1211 20:55:08.844324 (+ 108us) write_op.cc:270] Start()
1211 20:55:08.844338 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496357668 usec, L: 0
1211 20:55:08.844453 (+ 115us) write_op.cc:270] Start()
1211 20:55:08.844468 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496376419 usec, L: 0
1211 20:55:08.844570 (+ 102us) write_op.cc:270] Start()
1211 20:55:08.844584 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496391378 usec, L: 0
1211 20:55:08.844696 (+ 112us) write_op.cc:270] Start()
1211 20:55:08.844710 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496400520 usec, L: 0
1211 20:55:08.845834 (+ 1124us) write_op.cc:270] Start()
1211 20:55:08.845858 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496412696 usec, L: 0
1211 20:55:08.846042 (+ 184us) write_op.cc:270] Start()
1211 20:55:08.846063 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496415946 usec, L: 0
1211 20:55:08.846227 (+ 164us) write_op.cc:270] Start()
1211 20:55:08.846247 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496441811 usec, L: 0
1211 20:55:08.846374 (+ 127us) write_op.cc:270] Start()
1211 20:55:08.846394 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496448947 usec, L: 0
1211 20:55:08.846563 (+ 169us) write_op.cc:270] Start()
1211 20:55:08.846583 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496472174 usec, L: 0
1211 20:55:08.846735 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.846754 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496481779 usec, L: 0
1211 20:55:08.846912 (+ 158us) write_op.cc:270] Start()
1211 20:55:08.846957 (+ 45us) write_op.cc:276] Timestamp: P: 1765486496484365 usec, L: 0
1211 20:55:08.847145 (+ 188us) write_op.cc:270] Start()
1211 20:55:08.847166 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496493478 usec, L: 0
1211 20:55:08.847360 (+ 194us) write_op.cc:270] Start()
1211 20:55:08.847387 (+ 27us) write_op.cc:276] Timestamp: P: 1765486496508107 usec, L: 0
1211 20:55:08.847535 (+ 148us) write_op.cc:270] Start()
1211 20:55:08.847556 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496523148 usec, L: 0
1211 20:55:08.847699 (+ 143us) write_op.cc:270] Start()
1211 20:55:08.847719 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496528449 usec, L: 0
1211 20:55:08.853896 (+ 6177us) write_op.cc:270] Start()
1211 20:55:08.853916 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496534872 usec, L: 0
1211 20:55:08.903435 (+ 49519us) write_op.cc:270] Start()
1211 20:55:08.903469 (+ 34us) write_op.cc:276] Timestamp: P: 1765486496558391 usec, L: 0
1211 20:55:08.903611 (+ 142us) write_op.cc:270] Start()
1211 20:55:08.903630 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496570583 usec, L: 0
1211 20:55:08.903781 (+ 151us) write_op.cc:270] Start()
1211 20:55:08.903837 (+ 56us) write_op.cc:276] Timestamp: P: 1765486496576016 usec, L: 0
1211 20:55:08.904023 (+ 186us) write_op.cc:270] Start()
1211 20:55:08.904043 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496580266 usec, L: 0
1211 20:55:08.904194 (+ 151us) write_op.cc:270] Start()
1211 20:55:08.904220 (+ 26us) write_op.cc:276] Timestamp: P: 1765486496607321 usec, L: 0
1211 20:55:08.904364 (+ 144us) write_op.cc:270] Start()
1211 20:55:08.904384 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496626118 usec, L: 0
1211 20:55:08.904550 (+ 166us) write_op.cc:270] Start()
1211 20:55:08.904570 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496629226 usec, L: 0
1211 20:55:08.904702 (+ 132us) write_op.cc:270] Start()
1211 20:55:08.904719 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496641865 usec, L: 0
1211 20:55:08.906325 (+ 1606us) write_op.cc:270] Start()
1211 20:55:08.906341 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496649077 usec, L: 0
1211 20:55:08.913071 (+ 6730us) write_op.cc:270] Start()
1211 20:55:08.913094 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496670508 usec, L: 0
1211 20:55:08.926091 (+ 12997us) write_op.cc:270] Start()
1211 20:55:08.926115 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496689330 usec, L: 0
1211 20:55:08.932311 (+ 6196us) write_op.cc:270] Start()
1211 20:55:08.932326 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496694493 usec, L: 0
1211 20:55:08.950156 (+ 17830us) write_op.cc:270] Start()
1211 20:55:08.950181 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496696950 usec, L: 0
1211 20:55:08.950345 (+ 164us) write_op.cc:270] Start()
1211 20:55:08.950369 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496709796 usec, L: 0
1211 20:55:08.950560 (+ 191us) write_op.cc:270] Start()
1211 20:55:08.950582 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496744340 usec, L: 0
1211 20:55:08.950763 (+ 181us) write_op.cc:270] Start()
1211 20:55:08.950791 (+ 28us) write_op.cc:276] Timestamp: P: 1765486496753478 usec, L: 0
1211 20:55:08.950964 (+ 173us) write_op.cc:270] Start()
1211 20:55:08.950984 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496755788 usec, L: 0
1211 20:55:08.951167 (+ 183us) write_op.cc:270] Start()
1211 20:55:08.951191 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496779294 usec, L: 0
1211 20:55:08.987521 (+ 36330us) write_op.cc:270] Start()
1211 20:55:08.987540 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496799276 usec, L: 0
1211 20:55:08.987697 (+ 157us) write_op.cc:270] Start()
1211 20:55:08.987716 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496812978 usec, L: 0
1211 20:55:08.987836 (+ 120us) write_op.cc:270] Start()
1211 20:55:08.987853 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496814821 usec, L: 0
1211 20:55:08.987999 (+ 146us) write_op.cc:270] Start()
1211 20:55:08.988015 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496824174 usec, L: 0
1211 20:55:08.988162 (+ 147us) write_op.cc:270] Start()
1211 20:55:08.988178 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496826891 usec, L: 0
1211 20:55:08.988307 (+ 129us) write_op.cc:270] Start()
1211 20:55:08.988326 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496872946 usec, L: 0
1211 20:55:08.988491 (+ 165us) write_op.cc:270] Start()
1211 20:55:08.988509 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496880645 usec, L: 0
1211 20:55:08.988688 (+ 179us) write_op.cc:270] Start()
1211 20:55:08.988709 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496893871 usec, L: 0
1211 20:55:08.988847 (+ 138us) write_op.cc:270] Start()
1211 20:55:08.988864 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496910438 usec, L: 0
1211 20:55:08.989039 (+ 175us) write_op.cc:270] Start()
1211 20:55:08.989057 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496926932 usec, L: 0
1211 20:55:08.989207 (+ 150us) write_op.cc:270] Start()
1211 20:55:08.989226 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496939502 usec, L: 0
1211 20:55:08.989356 (+ 130us) write_op.cc:270] Start()
1211 20:55:08.989373 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496966507 usec, L: 0
1211 20:55:08.989509 (+ 136us) write_op.cc:270] Start()
1211 20:55:08.989527 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496973619 usec, L: 0
1211 20:55:08.989681 (+ 154us) write_op.cc:270] Start()
1211 20:55:08.989700 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496994225 usec, L: 0
1211 20:55:08.989856 (+ 156us) write_op.cc:270] Start()
1211 20:55:08.989874 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496995675 usec, L: 0
1211 20:55:09.008804 (+ 18930us) write_op.cc:270] Start()
1211 20:55:09.008828 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497003195 usec, L: 0
1211 20:55:09.012922 (+ 4094us) write_op.cc:270] Start()
1211 20:55:09.012947 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497027079 usec, L: 0
1211 20:55:09.013127 (+ 180us) write_op.cc:270] Start()
1211 20:55:09.013149 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497041392 usec, L: 0
1211 20:55:09.013300 (+ 151us) write_op.cc:270] Start()
1211 20:55:09.013328 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497057767 usec, L: 0
1211 20:55:09.013510 (+ 182us) write_op.cc:270] Start()
1211 20:55:09.013526 (+ 16us) write_op.cc:276] Timestamp: P: 1765486497079152 usec, L: 0
1211 20:55:09.013685 (+ 159us) write_op.cc:270] Start()
1211 20:55:09.013705 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497083840 usec, L: 0
1211 20:55:09.013853 (+ 148us) write_op.cc:270] Start()
1211 20:55:09.013872 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497085819 usec, L: 0
1211 20:55:09.014033 (+ 161us) write_op.cc:270] Start()
1211 20:55:09.014056 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497086452 usec, L: 0
1211 20:55:09.014200 (+ 144us) write_op.cc:270] Start()
1211 20:55:09.014222 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497115670 usec, L: 0
1211 20:55:09.014363 (+ 141us) write_op.cc:270] Start()
1211 20:55:09.014393 (+ 30us) write_op.cc:276] Timestamp: P: 1765486497132289 usec, L: 0
1211 20:55:09.014539 (+ 146us) write_op.cc:270] Start()
1211 20:55:09.014559 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497134801 usec, L: 0
1211 20:55:09.014693 (+ 134us) write_op.cc:270] Start()
1211 20:55:09.014712 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497139881 usec, L: 0
1211 20:55:09.014849 (+ 137us) write_op.cc:270] Start()
1211 20:55:09.014868 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497162725 usec, L: 0
1211 20:55:09.015008 (+ 140us) write_op.cc:270] Start()
1211 20:55:09.015026 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497176729 usec, L: 0
1211 20:55:09.015204 (+ 178us) write_op.cc:270] Start()
1211 20:55:09.015224 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497182648 usec, L: 0
1211 20:55:09.015373 (+ 149us) write_op.cc:270] Start()
1211 20:55:09.015402 (+ 29us) write_op.cc:276] Timestamp: P: 1765486497213039 usec, L: 0
1211 20:55:09.015554 (+ 152us) write_op.cc:270] Start()
1211 20:55:09.015572 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497219521 usec, L: 0
1211 20:55:09.015701 (+ 129us) write_op.cc:270] Start()
1211 20:55:09.015720 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497229985 usec, L: 0
1211 20:55:09.015883 (+ 163us) write_op.cc:270] Start()
1211 20:55:09.015901 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497237345 usec, L: 0
1211 20:55:09.016073 (+ 172us) write_op.cc:270] Start()
1211 20:55:09.016090 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497263781 usec, L: 0
1211 20:55:09.016214 (+ 124us) write_op.cc:270] Start()
1211 20:55:09.016230 (+ 16us) write_op.cc:276] Timestamp: P: 1765486497293003 usec, L: 0
1211 20:55:09.016359 (+ 129us) write_op.cc:270] Start()
1211 20:55:09.016409 (+ 50us) write_op.cc:276] Timestamp: P: 1765486497296818 usec, L: 0
1211 20:55:09.016546 (+ 137us) write_op.cc:270] Start()
1211 20:55:09.016563 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497304788 usec, L: 0
1211 20:55:09.016688 (+ 125us) write_op.cc:270] Start()
1211 20:55:09.016706 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497320626 usec, L: 0
1211 20:55:09.016853 (+ 147us) write_op.cc:270] Start()
1211 20:55:09.016870 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497352076 usec, L: 0
1211 20:55:09.016998 (+ 128us) write_op.cc:270] Start()
1211 20:55:09.017016 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497358764 usec, L: 0
1211 20:55:09.017170 (+ 154us) write_op.cc:270] Start()
1211 20:55:09.017188 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497366432 usec, L: 0
1211 20:55:09.017323 (+ 135us) write_op.cc:270] Start()
1211 20:55:09.017342 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497377355 usec, L: 0
1211 20:55:09.017501 (+ 159us) write_op.cc:270] Start()
1211 20:55:09.017519 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497396757 usec, L: 0
1211 20:55:09.017670 (+ 151us) write_op.cc:270] Start()
1211 20:55:09.017689 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497406538 usec, L: 0
1211 20:55:09.017838 (+ 149us) write_op.cc:270] Start()
1211 20:55:09.017859 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497421547 usec, L: 0
1211 20:55:09.051715 (+ 33856us) write_op.cc:270] Start()
1211 20:55:09.051738 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497444437 usec, L: 0
1211 20:55:09.062180 (+ 10442us) write_op.cc:270] Start()
1211 20:55:09.062210 (+ 30us) write_op.cc:276] Timestamp: P: 1765486497445211 usec, L: 0
1211 20:55:09.062417 (+ 207us) write_op.cc:270] Start()
1211 20:55:09.062439 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497451942 usec, L: 0
1211 20:55:09.062585 (+ 146us) write_op.cc:270] Start()
1211 20:55:09.062604 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497459933 usec, L: 0
1211 20:55:09.062760 (+ 156us) write_op.cc:270] Start()
1211 20:55:09.062781 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497489068 usec, L: 0
1211 20:55:09.062944 (+ 163us) write_op.cc:270] Start()
1211 20:55:09.062965 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497493600 usec, L: 0
1211 20:55:09.063143 (+ 178us) write_op.cc:270] Start()
1211 20:55:09.063166 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497495715 usec, L: 0
1211 20:55:09.063477 (+ 311us) write_op.cc:270] Start()
1211 20:55:09.063517 (+ 40us) write_op.cc:276] Timestamp: P: 1765486497505815 usec, L: 0
1211 20:55:09.063661 (+ 144us) write_op.cc:270] Start()
1211 20:55:09.063680 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497539915 usec, L: 0
1211 20:55:09.063838 (+ 158us) write_op.cc:270] Start()
1211 20:55:09.063857 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497549293 usec, L: 0
1211 20:55:09.064006 (+ 149us) write_op.cc:270] Start()
1211 20:55:09.064026 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497552733 usec, L: 0
1211 20:55:09.064182 (+ 156us) write_op.cc:270] Start()
1211 20:55:09.064202 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497555571 usec, L: 0
1211 20:55:09.064365 (+ 163us) write_op.cc:270] Start()
1211 20:55:09.064394 (+ 29us) write_op.cc:276] Timestamp: P: 1765486497584963 usec, L: 0
1211 20:55:09.064546 (+ 152us) write_op.cc:270] Start()
1211 20:55:09.064565 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497593285 usec, L: 0
1211 20:55:09.064710 (+ 145us) write_op.cc:270] Start()
1211 20:55:09.064728 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497609061 usec, L: 0
1211 20:55:09.100032 (+ 35304us) write_op.cc:270] Start()
1211 20:55:09.100060 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497619436 usec, L: 0
1211 20:55:09.100254 (+ 194us) write_op.cc:270] Start()
1211 20:55:09.100276 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497646525 usec, L: 0
1211 20:55:09.100462 (+ 186us) write_op.cc:270] Start()
1211 20:55:09.100485 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497653020 usec, L: 0
1211 20:55:09.100642 (+ 157us) write_op.cc:270] Start()
1211 20:55:09.100662 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497667352 usec, L: 0
1211 20:55:09.100832 (+ 170us) write_op.cc:270] Start()
1211 20:55:09.100852 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497680328 usec, L: 0
1211 20:55:09.100996 (+ 144us) write_op.cc:270] Start()
1211 20:55:09.101015 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497689646 usec, L: 0
1211 20:55:09.101150 (+ 135us) write_op.cc:270] Start()
1211 20:55:09.101173 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497708317 usec, L: 0
1211 20:55:09.101350 (+ 177us) write_op.cc:270] Start()
1211 20:55:09.101373 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497720777 usec, L: 0
1211 20:55:09.101595 (+ 222us) write_op.cc:270] Start()
1211 20:55:09.101630 (+ 35us) write_op.cc:276] Timestamp: P: 1765486497739906 usec, L: 0
1211 20:55:09.101798 (+ 168us) write_op.cc:270] Start()
1211 20:55:09.101819 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497751346 usec, L: 0
1211 20:55:09.101969 (+ 150us) write_op.cc:270] Start()
1211 20:55:09.102012 (+ 43us) write_op.cc:276] Timestamp: P: 1765486497761663 usec, L: 0
1211 20:55:09.102209 (+ 197us) write_op.cc:270] Start()
1211 20:55:09.102323 (+ 114us) write_op.cc:276] Timestamp: P: 1765486497777149 usec, L: 0
1211 20:55:09.102556 (+ 233us) write_op.cc:270] Start()
1211 20:55:09.102578 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497789658 usec, L: 0
1211 20:55:09.102741 (+ 163us) write_op.cc:270] Start()
1211 20:55:09.102764 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497803127 usec, L: 0
1211 20:55:09.102916 (+ 152us) write_op.cc:270] Start()
1211 20:55:09.102937 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497811776 usec, L: 0
1211 20:55:09.103093 (+ 156us) write_op.cc:270] Start()
1211 20:55:09.103117 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497820404 usec, L: 0
1211 20:55:09.103306 (+ 189us) write_op.cc:270] Start()
1211 20:55:09.103327 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497837989 usec, L: 0
1211 20:55:09.103498 (+ 171us) write_op.cc:270] Start()
1211 20:55:09.103518 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497855001 usec, L: 0
1211 20:55:09.103660 (+ 142us) write_op.cc:270] Start()
1211 20:55:09.103678 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497868119 usec, L: 0
1211 20:55:09.103824 (+ 146us) write_op.cc:270] Start()
1211 20:55:09.103843 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497879532 usec, L: 0
1211 20:55:09.103995 (+ 152us) write_op.cc:270] Start()
1211 20:55:09.104015 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497889008 usec, L: 0
1211 20:55:09.150035 (+ 46020us) write_op.cc:270] Start()
1211 20:55:09.150066 (+ 31us) write_op.cc:276] Timestamp: P: 1765486497910335 usec, L: 0
1211 20:55:09.150251 (+ 185us) write_op.cc:270] Start()
1211 20:55:09.150275 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497934905 usec, L: 0
1211 20:55:09.150495 (+ 220us) write_op.cc:270] Start()
1211 20:55:09.150515 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497957553 usec, L: 0
1211 20:55:09.150659 (+ 144us) write_op.cc:270] Start()
1211 20:55:09.150681 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497958134 usec, L: 0
1211 20:55:09.150823 (+ 142us) write_op.cc:270] Start()
1211 20:55:09.150842 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497959260 usec, L: 0
1211 20:55:09.150973 (+ 131us) write_op.cc:270] Start()
1211 20:55:09.150990 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497990029 usec, L: 0
1211 20:55:09.151117 (+ 127us) write_op.cc:270] Start()
1211 20:55:09.151135 (+ 18us) write_op.cc:276] Timestamp: P: 1765486498010017 usec, L: 0
1211 20:55:09.151306 (+ 171us) write_op.cc:270] Start()
1211 20:55:09.151325 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498028073 usec, L: 0
1211 20:55:09.151483 (+ 158us) write_op.cc:270] Start()
1211 20:55:09.151503 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498036075 usec, L: 0
1211 20:55:09.151640 (+ 137us) write_op.cc:270] Start()
1211 20:55:09.151661 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498042351 usec, L: 0
1211 20:55:09.151818 (+ 157us) write_op.cc:270] Start()
1211 20:55:09.151841 (+ 23us) write_op.cc:276] Timestamp: P: 1765486498049220 usec, L: 0
1211 20:55:09.152003 (+ 162us) write_op.cc:270] Start()
1211 20:55:09.152026 (+ 23us) write_op.cc:276] Timestamp: P: 1765486498080034 usec, L: 0
1211 20:55:09.152212 (+ 186us) write_op.cc:270] Start()
1211 20:55:09.152233 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498089803 usec, L: 0
1211 20:55:09.152424 (+ 191us) write_op.cc:270] Start()
1211 20:55:09.152443 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498111379 usec, L: 0
1211 20:55:09.152581 (+ 138us) write_op.cc:270] Start()
1211 20:55:09.152603 (+ 22us) write_op.cc:276] Timestamp: P: 1765486498115218 usec, L: 0
1211 20:55:09.152757 (+ 154us) write_op.cc:270] Start()
1211 20:55:09.152776 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498132109 usec, L: 0
1211 20:55:09.152923 (+ 147us) write_op.cc:270] Start()
1211 20:55:09.152942 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498136710 usec, L: 0
1211 20:55:09.153073 (+ 131us) write_op.cc:270] Start()
1211 20:55:09.153100 (+ 27us) write_op.cc:276] Timestamp: P: 1765486498159320 usec, L: 0
1211 20:55:09.153239 (+ 139us) write_op.cc:270] Start()
1211 20:55:09.153258 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498180466 usec, L: 0
1211 20:55:09.153420 (+ 162us) write_op.cc:270] Start()
1211 20:55:09.153450 (+ 30us) write_op.cc:276] Timestamp: P: 1765486498186130 usec, L: 0
1211 20:55:09.153601 (+ 151us) write_op.cc:270] Start()
1211 20:55:09.153620 (+ 19us) write_op.cc:276] Timestamp: P: 176548
W20251211 20:55:13.683715 6872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.211.194:45317, user_credentials={real_user=slave}} blocked reactor thread for 60724.8us
I20251211 20:55:13.684842 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:13.686789 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 881 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:13.871374 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:13.874351 7015 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:11.114780 6943 rpcz_store.cc:276] 1211 20:55:08.368368 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:08.372131 (+ 3763us) service_pool.cc:225] Handling call
1211 20:55:08.555990 (+183859us) raft_consensus.cc:1471] Updating replica for 721 ops
1211 20:55:08.684240 (+128250us) raft_consensus.cc:1534] Early marking committed up to index 0
1211 20:55:08.684245 (+ 5us) raft_consensus.cc:1539] Triggering prepare for 721 ops
1211 20:55:08.684552 (+ 307us) write_op.cc:270] Start()
1211 20:55:08.684583 (+ 31us) write_op.cc:276] Timestamp: P: 1765486495509592 usec, L: 0
1211 20:55:08.710972 (+ 26389us) write_op.cc:270] Start()
1211 20:55:08.711000 (+ 28us) write_op.cc:276] Timestamp: P: 1765486495534704 usec, L: 0
1211 20:55:08.712562 (+ 1562us) write_op.cc:270] Start()
1211 20:55:08.713034 (+ 472us) write_op.cc:276] Timestamp: P: 1765486495547590 usec, L: 0
1211 20:55:08.717726 (+ 4692us) write_op.cc:270] Start()
1211 20:55:08.717751 (+ 25us) write_op.cc:276] Timestamp: P: 1765486495551930 usec, L: 0
1211 20:55:08.719243 (+ 1492us) write_op.cc:270] Start()
1211 20:55:08.719274 (+ 31us) write_op.cc:276] Timestamp: P: 1765486495569328 usec, L: 0
1211 20:55:08.720347 (+ 1073us) write_op.cc:270] Start()
1211 20:55:08.720374 (+ 27us) write_op.cc:276] Timestamp: P: 1765486495584957 usec, L: 0
1211 20:55:08.721681 (+ 1307us) write_op.cc:270] Start()
1211 20:55:08.721727 (+ 46us) write_op.cc:276] Timestamp: P: 1765486495610147 usec, L: 0
1211 20:55:08.726319 (+ 4592us) write_op.cc:270] Start()
1211 20:55:08.726342 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495610992 usec, L: 0
1211 20:55:08.727616 (+ 1274us) write_op.cc:270] Start()
1211 20:55:08.728268 (+ 652us) write_op.cc:276] Timestamp: P: 1765486495625535 usec, L: 0
1211 20:55:08.729911 (+ 1643us) write_op.cc:270] Start()
1211 20:55:08.729934 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495642630 usec, L: 0
1211 20:55:08.731160 (+ 1226us) write_op.cc:270] Start()
1211 20:55:08.731189 (+ 29us) write_op.cc:276] Timestamp: P: 1765486495653752 usec, L: 0
1211 20:55:08.732383 (+ 1194us) write_op.cc:270] Start()
1211 20:55:08.732410 (+ 27us) write_op.cc:276] Timestamp: P: 1765486495678779 usec, L: 0
1211 20:55:08.733570 (+ 1160us) write_op.cc:270] Start()
1211 20:55:08.733593 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495692183 usec, L: 0
1211 20:55:08.734899 (+ 1306us) write_op.cc:270] Start()
1211 20:55:08.734924 (+ 25us) write_op.cc:276] Timestamp: P: 1765486495692818 usec, L: 0
1211 20:55:08.735970 (+ 1046us) write_op.cc:270] Start()
1211 20:55:08.735995 (+ 25us) write_op.cc:276] Timestamp: P: 1765486495722028 usec, L: 0
1211 20:55:08.737410 (+ 1415us) write_op.cc:270] Start()
1211 20:55:08.737432 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495725720 usec, L: 0
1211 20:55:08.740529 (+ 3097us) write_op.cc:270] Start()
1211 20:55:08.740561 (+ 32us) write_op.cc:276] Timestamp: P: 1765486495744318 usec, L: 0
1211 20:55:08.742627 (+ 2066us) write_op.cc:270] Start()
1211 20:55:08.742652 (+ 25us) write_op.cc:276] Timestamp: P: 1765486495764856 usec, L: 0
1211 20:55:08.744343 (+ 1691us) write_op.cc:270] Start()
1211 20:55:08.746478 (+ 2135us) write_op.cc:276] Timestamp: P: 1765486495777505 usec, L: 0
1211 20:55:08.747463 (+ 985us) write_op.cc:270] Start()
1211 20:55:08.747486 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495798829 usec, L: 0
1211 20:55:08.748910 (+ 1424us) write_op.cc:270] Start()
1211 20:55:08.748938 (+ 28us) write_op.cc:276] Timestamp: P: 1765486495801255 usec, L: 0
1211 20:55:08.750542 (+ 1604us) write_op.cc:270] Start()
1211 20:55:08.750568 (+ 26us) write_op.cc:276] Timestamp: P: 1765486495837691 usec, L: 0
1211 20:55:08.751965 (+ 1397us) write_op.cc:270] Start()
1211 20:55:08.751986 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495859730 usec, L: 0
1211 20:55:08.753162 (+ 1176us) write_op.cc:270] Start()
1211 20:55:08.753185 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495880711 usec, L: 0
1211 20:55:08.754650 (+ 1465us) write_op.cc:270] Start()
1211 20:55:08.754691 (+ 41us) write_op.cc:276] Timestamp: P: 1765486495892583 usec, L: 0
1211 20:55:08.756573 (+ 1882us) write_op.cc:270] Start()
1211 20:55:08.756607 (+ 34us) write_op.cc:276] Timestamp: P: 1765486495894046 usec, L: 0
1211 20:55:08.758168 (+ 1561us) write_op.cc:270] Start()
1211 20:55:08.758195 (+ 27us) write_op.cc:276] Timestamp: P: 1765486495911603 usec, L: 0
1211 20:55:08.768361 (+ 10166us) write_op.cc:270] Start()
1211 20:55:08.768403 (+ 42us) write_op.cc:276] Timestamp: P: 1765486495931751 usec, L: 0
1211 20:55:08.769874 (+ 1471us) write_op.cc:270] Start()
1211 20:55:08.769896 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495953612 usec, L: 0
1211 20:55:08.770049 (+ 153us) write_op.cc:270] Start()
1211 20:55:08.770084 (+ 35us) write_op.cc:276] Timestamp: P: 1765486495958186 usec, L: 0
1211 20:55:08.770220 (+ 136us) write_op.cc:270] Start()
1211 20:55:08.770237 (+ 17us) write_op.cc:276] Timestamp: P: 1765486495961944 usec, L: 0
1211 20:55:08.770367 (+ 130us) write_op.cc:270] Start()
1211 20:55:08.770385 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495972366 usec, L: 0
1211 20:55:08.770564 (+ 179us) write_op.cc:270] Start()
1211 20:55:08.770582 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495996042 usec, L: 0
1211 20:55:08.770761 (+ 179us) write_op.cc:270] Start()
1211 20:55:08.770786 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496009502 usec, L: 0
1211 20:55:08.770938 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.770955 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496019183 usec, L: 0
1211 20:55:08.771113 (+ 158us) write_op.cc:270] Start()
1211 20:55:08.771148 (+ 35us) write_op.cc:276] Timestamp: P: 1765486496030525 usec, L: 0
1211 20:55:08.771289 (+ 141us) write_op.cc:270] Start()
1211 20:55:08.771379 (+ 90us) write_op.cc:276] Timestamp: P: 1765486496057883 usec, L: 0
1211 20:55:08.771540 (+ 161us) write_op.cc:270] Start()
1211 20:55:08.771594 (+ 54us) write_op.cc:276] Timestamp: P: 1765486496059843 usec, L: 0
1211 20:55:08.771776 (+ 182us) write_op.cc:270] Start()
1211 20:55:08.771796 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496070764 usec, L: 0
1211 20:55:08.771961 (+ 165us) write_op.cc:270] Start()
1211 20:55:08.772001 (+ 40us) write_op.cc:276] Timestamp: P: 1765486496082015 usec, L: 0
1211 20:55:08.772154 (+ 153us) write_op.cc:270] Start()
1211 20:55:08.772173 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496098171 usec, L: 0
1211 20:55:08.772325 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.772345 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496114579 usec, L: 0
1211 20:55:08.772489 (+ 144us) write_op.cc:270] Start()
1211 20:55:08.772508 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496123534 usec, L: 0
1211 20:55:08.772664 (+ 156us) write_op.cc:270] Start()
1211 20:55:08.772682 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496132657 usec, L: 0
1211 20:55:08.772869 (+ 187us) write_op.cc:270] Start()
1211 20:55:08.772889 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496138853 usec, L: 0
1211 20:55:08.773057 (+ 168us) write_op.cc:270] Start()
1211 20:55:08.773088 (+ 31us) write_op.cc:276] Timestamp: P: 1765486496155054 usec, L: 0
1211 20:55:08.773240 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.773259 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496162858 usec, L: 0
1211 20:55:08.773431 (+ 172us) write_op.cc:270] Start()
1211 20:55:08.773449 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496190606 usec, L: 0
1211 20:55:08.773582 (+ 133us) write_op.cc:270] Start()
1211 20:55:08.773599 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496191821 usec, L: 0
1211 20:55:08.798396 (+ 24797us) write_op.cc:270] Start()
1211 20:55:08.798421 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496196767 usec, L: 0
1211 20:55:08.804935 (+ 6514us) write_op.cc:270] Start()
1211 20:55:08.804964 (+ 29us) write_op.cc:276] Timestamp: P: 1765486496216370 usec, L: 0
1211 20:55:08.805161 (+ 197us) write_op.cc:270] Start()
1211 20:55:08.805184 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496228489 usec, L: 0
1211 20:55:08.805336 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.805358 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496241266 usec, L: 0
1211 20:55:08.805467 (+ 109us) write_op.cc:270] Start()
1211 20:55:08.805480 (+ 13us) write_op.cc:276] Timestamp: P: 1765486496263443 usec, L: 0
1211 20:55:08.805635 (+ 155us) write_op.cc:270] Start()
1211 20:55:08.805656 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496286734 usec, L: 0
1211 20:55:08.805803 (+ 147us) write_op.cc:270] Start()
1211 20:55:08.805845 (+ 42us) write_op.cc:276] Timestamp: P: 1765486496291827 usec, L: 0
1211 20:55:08.806096 (+ 251us) write_op.cc:270] Start()
1211 20:55:08.806115 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496294405 usec, L: 0
1211 20:55:08.814279 (+ 8164us) write_op.cc:270] Start()
1211 20:55:08.814320 (+ 41us) write_op.cc:276] Timestamp: P: 1765486496315706 usec, L: 0
1211 20:55:08.814864 (+ 544us) write_op.cc:270] Start()
1211 20:55:08.814886 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496338514 usec, L: 0
1211 20:55:08.826219 (+ 11333us) write_op.cc:270] Start()
1211 20:55:08.827250 (+ 1031us) write_op.cc:276] Timestamp: P: 1765486496344955 usec, L: 0
1211 20:55:08.827371 (+ 121us) write_op.cc:270] Start()
1211 20:55:08.827389 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496356739 usec, L: 0
1211 20:55:08.827502 (+ 113us) write_op.cc:270] Start()
1211 20:55:08.827518 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496377112 usec, L: 0
1211 20:55:08.827653 (+ 135us) write_op.cc:270] Start()
1211 20:55:08.827670 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496392644 usec, L: 0
1211 20:55:08.827787 (+ 117us) write_op.cc:270] Start()
1211 20:55:08.827803 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496398767 usec, L: 0
1211 20:55:08.828716 (+ 913us) write_op.cc:270] Start()
1211 20:55:08.828735 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496413796 usec, L: 0
1211 20:55:08.829228 (+ 493us) write_op.cc:270] Start()
1211 20:55:08.829247 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496418961 usec, L: 0
1211 20:55:08.829604 (+ 357us) write_op.cc:270] Start()
1211 20:55:08.829621 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496438869 usec, L: 0
1211 20:55:08.829822 (+ 201us) write_op.cc:270] Start()
1211 20:55:08.829839 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496448898 usec, L: 0
1211 20:55:08.830158 (+ 319us) write_op.cc:270] Start()
1211 20:55:08.830178 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496470166 usec, L: 0
1211 20:55:08.830296 (+ 118us) write_op.cc:270] Start()
1211 20:55:08.830312 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496475496 usec, L: 0
1211 20:55:08.830425 (+ 113us) write_op.cc:270] Start()
1211 20:55:08.830440 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496483127 usec, L: 0
1211 20:55:08.830544 (+ 104us) write_op.cc:270] Start()
1211 20:55:08.830567 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496495230 usec, L: 0
1211 20:55:08.830942 (+ 375us) write_op.cc:270] Start()
1211 20:55:08.830958 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496507429 usec, L: 0
1211 20:55:08.831088 (+ 130us) write_op.cc:270] Start()
1211 20:55:08.831104 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496524689 usec, L: 0
1211 20:55:08.831207 (+ 103us) write_op.cc:270] Start()
1211 20:55:08.831222 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496530109 usec, L: 0
1211 20:55:08.831318 (+ 96us) write_op.cc:270] Start()
1211 20:55:08.831333 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496540623 usec, L: 0
1211 20:55:08.832737 (+ 1404us) write_op.cc:270] Start()
1211 20:55:08.832761 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496558777 usec, L: 0
1211 20:55:08.832935 (+ 174us) write_op.cc:270] Start()
1211 20:55:08.832954 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496567502 usec, L: 0
1211 20:55:08.833071 (+ 117us) write_op.cc:270] Start()
1211 20:55:08.833162 (+ 91us) write_op.cc:276] Timestamp: P: 1765486496578043 usec, L: 0
1211 20:55:08.833346 (+ 184us) write_op.cc:270] Start()
1211 20:55:08.833368 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496580274 usec, L: 0
1211 20:55:08.833547 (+ 179us) write_op.cc:270] Start()
1211 20:55:08.833567 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496603974 usec, L: 0
1211 20:55:08.833725 (+ 158us) write_op.cc:270] Start()
1211 20:55:08.833744 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496629488 usec, L: 0
1211 20:55:08.833891 (+ 147us) write_op.cc:270] Start()
1211 20:55:08.833911 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496632308 usec, L: 0
1211 20:55:08.834084 (+ 173us) write_op.cc:270] Start()
1211 20:55:08.834107 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496636827 usec, L: 0
1211 20:55:08.834281 (+ 174us) write_op.cc:270] Start()
1211 20:55:08.834301 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496649500 usec, L: 0
1211 20:55:08.834484 (+ 183us) write_op.cc:270] Start()
1211 20:55:08.834504 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496669056 usec, L: 0
1211 20:55:08.834978 (+ 474us) write_op.cc:270] Start()
1211 20:55:08.835000 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496689420 usec, L: 0
1211 20:55:08.840768 (+ 5768us) write_op.cc:270] Start()
1211 20:55:08.840797 (+ 29us) write_op.cc:276] Timestamp: P: 1765486496692828 usec, L: 0
1211 20:55:08.840940 (+ 143us) write_op.cc:270] Start()
1211 20:55:08.840962 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496697241 usec, L: 0
1211 20:55:08.841092 (+ 130us) write_op.cc:270] Start()
1211 20:55:08.841108 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496715246 usec, L: 0
1211 20:55:08.841252 (+ 144us) write_op.cc:270] Start()
1211 20:55:08.841267 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496741520 usec, L: 0
1211 20:55:08.841380 (+ 113us) write_op.cc:270] Start()
1211 20:55:08.842105 (+ 725us) write_op.cc:276] Timestamp: P: 1765486496756977 usec, L: 0
1211 20:55:08.842253 (+ 148us) write_op.cc:270] Start()
1211 20:55:08.842275 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496757565 usec, L: 0
1211 20:55:08.842388 (+ 113us) write_op.cc:270] Start()
1211 20:55:08.842404 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496787517 usec, L: 0
1211 20:55:08.842505 (+ 101us) write_op.cc:270] Start()
1211 20:55:08.842519 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496789674 usec, L: 0
1211 20:55:08.842657 (+ 138us) write_op.cc:270] Start()
1211 20:55:08.842672 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496814129 usec, L: 0
1211 20:55:08.842770 (+ 98us) write_op.cc:270] Start()
1211 20:55:08.842784 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496817270 usec, L: 0
1211 20:55:08.842901 (+ 117us) write_op.cc:270] Start()
1211 20:55:08.842916 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496823430 usec, L: 0
1211 20:55:08.843017 (+ 101us) write_op.cc:270] Start()
1211 20:55:08.843031 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496853934 usec, L: 0
1211 20:55:08.843143 (+ 112us) write_op.cc:270] Start()
1211 20:55:08.843158 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496865749 usec, L: 0
1211 20:55:08.843259 (+ 101us) write_op.cc:270] Start()
1211 20:55:08.843273 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496874300 usec, L: 0
1211 20:55:08.843430 (+ 157us) write_op.cc:270] Start()
1211 20:55:08.843445 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496894631 usec, L: 0
1211 20:55:08.851005 (+ 7560us) write_op.cc:270] Start()
1211 20:55:08.851028 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496909263 usec, L: 0
1211 20:55:08.851205 (+ 177us) write_op.cc:270] Start()
1211 20:55:08.851224 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496926256 usec, L: 0
1211 20:55:08.851375 (+ 151us) write_op.cc:270] Start()
1211 20:55:08.851412 (+ 37us) write_op.cc:276] Timestamp: P: 1765486496931839 usec, L: 0
1211 20:55:08.851552 (+ 140us) write_op.cc:270] Start()
1211 20:55:08.851571 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496963249 usec, L: 0
1211 20:55:08.851712 (+ 141us) write_op.cc:270] Start()
1211 20:55:08.851731 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496970305 usec, L: 0
1211 20:55:08.851874 (+ 143us) write_op.cc:270] Start()
1211 20:55:08.851894 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496992317 usec, L: 0
1211 20:55:08.852059 (+ 165us) write_op.cc:270] Start()
1211 20:55:08.852098 (+ 39us) write_op.cc:276] Timestamp: P: 1765486496998143 usec, L: 0
1211 20:55:08.852271 (+ 173us) write_op.cc:270] Start()
1211 20:55:08.852290 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496998733 usec, L: 0
1211 20:55:08.852432 (+ 142us) write_op.cc:270] Start()
1211 20:55:08.852451 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497019436 usec, L: 0
1211 20:55:08.852609 (+ 158us) write_op.cc:270] Start()
1211 20:55:08.852631 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497040174 usec, L: 0
1211 20:55:08.852797 (+ 166us) write_op.cc:270] Start()
1211 20:55:08.852818 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497059338 usec, L: 0
1211 20:55:08.852956 (+ 138us) write_op.cc:270] Start()
1211 20:55:08.852975 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497075349 usec, L: 0
1211 20:55:08.853131 (+ 156us) write_op.cc:270] Start()
1211 20:55:08.853149 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497081000 usec, L: 0
1211 20:55:08.888684 (+ 35535us) write_op.cc:270] Start()
1211 20:55:08.888704 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497086192 usec, L: 0
1211 20:55:08.906481 (+ 17777us) write_op.cc:270] Start()
1211 20:55:08.906500 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497089109 usec, L: 0
1211 20:55:08.913244 (+ 6744us) write_op.cc:270] Start()
1211 20:55:08.913267 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497119826 usec, L: 0
1211 20:55:08.914372 (+ 1105us) write_op.cc:270] Start()
1211 20:55:08.914410 (+ 38us) write_op.cc:276] Timestamp: P: 1765486497138335 usec, L: 0
1211 20:55:08.928185 (+ 13775us) write_op.cc:270] Start()
1211 20:55:08.928227 (+ 42us) write_op.cc:276] Timestamp: P: 1765486497139282 usec, L: 0
1211 20:55:08.930612 (+ 2385us) write_op.cc:270] Start()
1211 20:55:08.930640 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497141734 usec, L: 0
1211 20:55:08.932217 (+ 1577us) write_op.cc:270] Start()
1211 20:55:08.932240 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497163369 usec, L: 0
1211 20:55:08.940008 (+ 7768us) write_op.cc:270] Start()
1211 20:55:08.940033 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497178668 usec, L: 0
1211 20:55:08.940320 (+ 287us) write_op.cc:270] Start()
1211 20:55:08.940353 (+ 33us) write_op.cc:276] Timestamp: P: 1765486497184045 usec, L: 0
1211 20:55:08.940525 (+ 172us) write_op.cc:270] Start()
1211 20:55:08.954132 (+ 13607us) write_op.cc:276] Timestamp: P: 1765486497209703 usec, L: 0
1211 20:55:08.954297 (+ 165us) write_op.cc:270] Start()
1211 20:55:08.954315 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497224470 usec, L: 0
1211 20:55:08.962078 (+ 7763us) write_op.cc:270] Start()
1211 20:55:08.962107 (+ 29us) write_op.cc:276] Timestamp: P: 1765486497232218 usec, L: 0
1211 20:55:08.962368 (+ 261us) write_op.cc:270] Start()
1211 20:55:08.962390 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497236024 usec, L: 0
1211 20:55:08.962565 (+ 175us) write_op.cc:270] Start()
1211 20:55:08.962586 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497265248 usec, L: 0
1211 20:55:08.962716 (+ 130us) write_op.cc:270] Start()
1211 20:55:08.962734 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497292328 usec, L: 0
1211 20:55:08.962975 (+ 241us) write_op.cc:270] Start()
1211 20:55:08.963021 (+ 46us) write_op.cc:276] Timestamp: P: 1765486497296292 usec, L: 0
1211 20:55:08.963211 (+ 190us) write_op.cc:270] Start()
1211 20:55:08.963239 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497300507 usec, L: 0
1211 20:55:08.963381 (+ 142us) write_op.cc:270] Start()
1211 20:55:08.963402 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497331576 usec, L: 0
1211 20:55:08.963581 (+ 179us) write_op.cc:270] Start()
1211 20:55:08.963602 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497353850 usec, L: 0
1211 20:55:08.963737 (+ 135us) write_op.cc:270] Start()
1211 20:55:08.963757 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497360227 usec, L: 0
1211 20:55:08.963924 (+ 167us) write_op.cc:270] Start()
1211 20:55:08.963952 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497373175 usec, L: 0
1211 20:55:08.964146 (+ 194us) write_op.cc:270] Start()
1211 20:55:08.964169 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497381016 usec, L: 0
1211 20:55:08.964318 (+ 149us) write_op.cc:270] Start()
1211 20:55:08.964335 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497397516 usec, L: 0
1211 20:55:08.964472 (+ 137us) write_op.cc:270] Start()
1211 20:55:08.964491 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497407125 usec, L: 0
1211 20:55:08.964658 (+ 167us) write_op.cc:270] Start()
1211 20:55:08.964688 (+ 30us) write_op.cc:276] Timestamp: P: 1765486497411367 usec, L: 0
1211 20:55:08.964926 (+ 238us) write_op.cc:270] Start()
1211 20:55:08.964949 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497443704 usec, L: 0
1211 20:55:08.965101 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.965119 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497444453 usec, L: 0
1211 20:55:08.965267 (+ 148us) write_op.cc:270] Start()
1211 20:55:09.008895 (+ 43628us) write_op.cc:276] Timestamp: P: 1765486497450538 usec, L: 0
1211 20:55:09.009054 (+ 159us) write_op.cc:270] Start()
1211 20:55:09.009077 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497460286 usec, L: 0
1211 20:55:09.009237 (+ 160us) write_op.cc:270] Start()
1211 20:55:09.009256 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497488663 usec, L: 0
1211 20:55:09.009426 (+ 170us) write_op.cc:270] Start()
1211 20:55:09.009445 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497499596 usec, L: 0
1211 20:55:09.009593 (+ 148us) write_op.cc:270] Start()
1211 20:55:09.009616 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497510922 usec, L: 0
1211 20:55:09.009763 (+ 147us) write_op.cc:270] Start()
1211 20:55:09.009781 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497517422 usec, L: 0
1211 20:55:09.009928 (+ 147us) write_op.cc:270] Start()
1211 20:55:09.009947 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497538781 usec, L: 0
1211 20:55:09.010139 (+ 192us) write_op.cc:270] Start()
1211 20:55:09.010159 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497541443 usec, L: 0
1211 20:55:09.010309 (+ 150us) write_op.cc:270] Start()
1211 20:55:09.010329 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497554754 usec, L: 0
1211 20:55:09.010488 (+ 159us) write_op.cc:270] Start()
1211 20:55:09.010508 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497559384 usec, L: 0
1211 20:55:09.010687 (+ 179us) write_op.cc:270] Start()
1211 20:55:09.010707 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497578933 usec, L: 0
1211 20:55:09.010830 (+ 123us) write_op.cc:270] Start()
1211 20:55:09.010850 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497598726 usec, L: 0
1211 20:55:09.010996 (+ 146us) write_op.cc:270] Start()
1211 20:55:09.011014 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497607493 usec, L: 0
1211 20:55:09.011191 (+ 177us) write_op.cc:270] Start()
1211 20:55:09.011210 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497620808 usec, L: 0
1211 20:55:09.011336 (+ 126us) write_op.cc:270] Start()
1211 20:55:09.011362 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497648780 usec, L: 0
1211 20:55:09.011573 (+ 211us) write_op.cc:270] Start()
1211 20:55:09.011595 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497655554 usec, L: 0
1211 20:55:09.011734 (+ 139us) write_op.cc:270] Start()
1211 20:55:09.011753 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497661697 usec, L: 0
1211 20:55:09.011878 (+ 125us) write_op.cc:270] Start()
1211 20:55:09.011896 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497681242 usec, L: 0
1211 20:55:09.012049 (+ 153us) write_op.cc:270] Start()
1211 20:55:09.012068 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497685120 usec, L: 0
1211 20:55:09.012261 (+ 193us) write_op.cc:270] Start()
1211 20:55:09.012283 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497709640 usec, L: 0
1211 20:55:09.012450 (+ 167us) write_op.cc:270] Start()
1211 20:55:09.012470 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497733127 usec, L: 0
1211 20:55:09.050043 (+ 37573us) write_op.cc:270] Start()
1211 20:55:09.050065 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497737260 usec, L: 0
1211 20:55:09.050222 (+ 157us) write_op.cc:270] Start()
1211 20:55:09.050241 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497754136 usec, L: 0
1211 20:55:09.050428 (+ 187us) write_op.cc:270] Start()
1211 20:55:09.050457 (+ 29us) write_op.cc:276] Timestamp: P: 1765486497759350 usec, L: 0
1211 20:55:09.050607 (+ 150us) write_op.cc:270] Start()
1211 20:55:09.050685 (+ 78us) write_op.cc:276] Timestamp: P: 1765486497777157 usec, L: 0
1211 20:55:09.050990 (+ 305us) write_op.cc:270] Start()
1211 20:55:09.051011 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497782088 usec, L: 0
1211 20:55:09.051152 (+ 141us) write_op.cc:270] Start()
1211 20:55:09.051170 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497805359 usec, L: 0
1211 20:55:09.051297 (+ 127us) write_op.cc:270] Start()
1211 20:55:09.051313 (+ 16us) write_op.cc:276] Timestamp: P: 1765486497815865 usec, L: 0
1211 20:55:09.051443 (+ 130us) write_op.cc:270] Start()
1211 20:55:09.051457 (+ 14us) write_op.cc:276] Timestamp: P: 1765486497818479 usec, L: 0
1211 20:55:09.051560 (+ 103us) write_op.cc:270] Start()
1211 20:55:09.051575 (+ 15us) write_op.cc:276] Timestamp: P: 1765486497835298 usec, L: 0
1211 20:55:09.078127 (+ 26552us) write_op.cc:270] Start()
1211 20:55:09.078155 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497854713 usec, L: 0
1211 20:55:09.078325 (+ 170us) write_op.cc:270] Start()
1211 20:55:09.078347 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497866523 usec, L: 0
1211 20:55:09.078524 (+ 177us) write_op.cc:270] Start()
1211 20:55:09.078545 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497885902 usec, L: 0
1211 20:55:09.078697 (+ 152us) write_op.cc:270] Start()
1211 20:55:09.078717 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497889129 usec, L: 0
1211 20:55:09.078867 (+ 150us) write_op.cc:270] Start()
1211 20:55:09.078889 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497896834 usec, L: 0
1211 20:55:09.079037 (+ 148us) write_op.cc:270] Start()
1211 20:55:09.079058 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497932066 usec, L: 0
1211 20:55:09.079257 (+ 199us) write_op.cc:270] Start()
1211 20:55:09.079278 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497950802 usec, L: 0
1211 20:55:09.079478 (+ 200us) write_op.cc:270] Start()
1211 20:55:09.079498 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497954413 usec, L: 0
1211 20:55:09.079633 (+ 135us) write_op.cc:270] Start()
1211 20:55:09.079651 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497960682 usec, L: 0
1211 20:55:09.079794 (+ 143us) write_op.cc:270] Start()
1211 20:55:09.079812 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497988978 usec, L: 0
1211 20:55:09.079948 (+ 136us) write_op.cc:270] Start()
1211 20:55:09.079966 (+ 18us) write_op.cc:276] Timestamp: P: 1765486498006783 usec, L: 0
1211 20:55:09.080210 (+ 244us) write_op.cc:270] Start()
1211 20:55:09.080230 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498033918 usec, L: 0
1211 20:55:09.080383 (+ 153us) write_op.cc:270] Start()
1211 20:55:09.080402 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498035936 usec, L: 0
1211 20:55:09.080527 (+ 125us) write_op.cc:270] Start()
1211 20:55:09.080546 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498041583 usec, L: 0
1211 20:55:09.080735 (+ 189us) write_op.cc:270] Start()
1211 20:55:09.080754 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498047154 usec, L: 0
1211 20:55:09.080910 (+ 156us) write_op.cc:270] Start()
1211 20:55:09.080931 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498080046 usec, L: 0
1211 20:55:09.081070 (+ 139us) write_op.cc:270] Start()
1211 20:55:09.081089 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498092824 usec, L: 0
1211 20:55:09.081261 (+ 172us) write_op.cc:270] Start()
1211 20:55:09.081283 (+ 22us) write_op.cc:276] Timestamp: P: 1765486498107386 usec, L: 0
1211 20:55:09.081443 (+ 160us) write_op.cc:270] Start()
1211 20:55:09.081463 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498117042 usec, L: 0
1211 20:55:09.081600 (+ 137us) write_op.cc:270] Start()
1211 20:55:09.081619 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498130208 usec, L: 0
1211 20:55:09.081754 (+ 135us) write_op.cc:270] Start()
1211 20:55:09.081775 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498137404 usec, L: 0
1211 20:55:09.081906 (+ 131us) write_op.cc:270] Start()
1211 20:55:09.081934 (+ 28us) write_op.cc:276] Timestamp: P: 1765486498161212 usec, L: 0
1211 20:55:09.129739 (+ 47805us) write_op.cc:270] Start()
1211 20:55:09.129761 (+ 22us) write_op.cc:276] Timestamp: P: 1765486498179703 usec, L: 0
1211 20:55:09.129945 (+ 184us) write_op.cc:270] Start()
1211 20:55:09.129965 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498183241 usec, L: 0
1211 20:55:09.130152 (+ 187us) write_op.cc:270] Start()
1211 20:55:09.130174 (+ 22us) write_op.cc:276] Timestamp: P: 17654864
I20251211 20:55:14.416465 7051 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
W20251211 20:55:14.416469 7049 log.cc:927] Time spent T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
I20251211 20:55:14.419283 7051 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:14.420670 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1145 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:14.420912 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1145 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:14.422015 7051 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:14.433507 6872 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:14.440554 7051 raft_consensus.cc:2749] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:14.504012 6947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.195:38469 (request call id 11) took 1386 ms. Trace:
I20251211 20:55:14.504760 6947 rpcz_store.cc:276] 1211 20:55:13.116783 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:13.117382 (+ 599us) service_pool.cc:225] Handling call
1211 20:55:13.326268 (+208886us) raft_consensus.cc:1471] Updating replica for 235 ops
1211 20:55:13.512321 (+186053us) raft_consensus.cc:1534] Early marking committed up to index 0
1211 20:55:13.512326 (+ 5us) raft_consensus.cc:1539] Triggering prepare for 235 ops
1211 20:55:13.518988 (+ 6662us) write_op.cc:270] Start()
1211 20:55:13.519013 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508635948 usec, L: 0
1211 20:55:13.523783 (+ 4770us) write_op.cc:270] Start()
1211 20:55:13.523808 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508639712 usec, L: 0
1211 20:55:13.524031 (+ 223us) write_op.cc:270] Start()
1211 20:55:13.524062 (+ 31us) write_op.cc:276] Timestamp: P: 1765486508654358 usec, L: 0
1211 20:55:13.524230 (+ 168us) write_op.cc:270] Start()
1211 20:55:13.524256 (+ 26us) write_op.cc:276] Timestamp: P: 1765486508669954 usec, L: 0
1211 20:55:13.524444 (+ 188us) write_op.cc:270] Start()
1211 20:55:13.524466 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508712374 usec, L: 0
1211 20:55:13.527679 (+ 3213us) write_op.cc:270] Start()
1211 20:55:13.527698 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508727407 usec, L: 0
1211 20:55:13.530104 (+ 2406us) write_op.cc:270] Start()
1211 20:55:13.530133 (+ 29us) write_op.cc:276] Timestamp: P: 1765486508749009 usec, L: 0
1211 20:55:13.530273 (+ 140us) write_op.cc:270] Start()
1211 20:55:13.530288 (+ 15us) write_op.cc:276] Timestamp: P: 1765486508779897 usec, L: 0
1211 20:55:13.532874 (+ 2586us) write_op.cc:270] Start()
1211 20:55:13.532907 (+ 33us) write_op.cc:276] Timestamp: P: 1765486508817173 usec, L: 0
1211 20:55:13.533085 (+ 178us) write_op.cc:270] Start()
1211 20:55:13.533107 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508818386 usec, L: 0
1211 20:55:13.534169 (+ 1062us) write_op.cc:270] Start()
1211 20:55:13.534191 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508819068 usec, L: 0
1211 20:55:13.557171 (+ 22980us) write_op.cc:270] Start()
1211 20:55:13.557199 (+ 28us) write_op.cc:276] Timestamp: P: 1765486508876016 usec, L: 0
1211 20:55:13.573242 (+ 16043us) write_op.cc:270] Start()
1211 20:55:13.573264 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508899392 usec, L: 0
1211 20:55:13.577712 (+ 4448us) write_op.cc:270] Start()
1211 20:55:13.577735 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508908417 usec, L: 0
1211 20:55:13.577921 (+ 186us) write_op.cc:270] Start()
1211 20:55:13.577952 (+ 31us) write_op.cc:276] Timestamp: P: 1765486508909675 usec, L: 0
1211 20:55:13.578127 (+ 175us) write_op.cc:270] Start()
1211 20:55:13.578148 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508966433 usec, L: 0
1211 20:55:13.579374 (+ 1226us) write_op.cc:270] Start()
1211 20:55:13.579398 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509001455 usec, L: 0
1211 20:55:13.579563 (+ 165us) write_op.cc:270] Start()
1211 20:55:13.579600 (+ 37us) write_op.cc:276] Timestamp: P: 1765486509016090 usec, L: 0
1211 20:55:13.579778 (+ 178us) write_op.cc:270] Start()
1211 20:55:13.579801 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509035126 usec, L: 0
1211 20:55:13.580889 (+ 1088us) write_op.cc:270] Start()
1211 20:55:13.582129 (+ 1240us) write_op.cc:276] Timestamp: P: 1765486509056852 usec, L: 0
1211 20:55:13.582311 (+ 182us) write_op.cc:270] Start()
1211 20:55:13.582332 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509058665 usec, L: 0
1211 20:55:13.582489 (+ 157us) write_op.cc:270] Start()
1211 20:55:13.582507 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509101482 usec, L: 0
1211 20:55:13.582642 (+ 135us) write_op.cc:270] Start()
1211 20:55:13.582666 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509125482 usec, L: 0
1211 20:55:13.582814 (+ 148us) write_op.cc:270] Start()
1211 20:55:13.582833 (+ 19us) write_op.cc:276] Timestamp: P: 1765486509164634 usec, L: 0
1211 20:55:13.585281 (+ 2448us) write_op.cc:270] Start()
1211 20:55:13.585308 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509165220 usec, L: 0
1211 20:55:13.587353 (+ 2045us) write_op.cc:270] Start()
1211 20:55:13.587378 (+ 25us) write_op.cc:276] Timestamp: P: 1765486509197140 usec, L: 0
1211 20:55:13.596817 (+ 9439us) write_op.cc:270] Start()
1211 20:55:13.596853 (+ 36us) write_op.cc:276] Timestamp: P: 1765486509215631 usec, L: 0
1211 20:55:13.619994 (+ 23141us) write_op.cc:270] Start()
1211 20:55:13.620027 (+ 33us) write_op.cc:276] Timestamp: P: 1765486509220701 usec, L: 0
1211 20:55:13.633530 (+ 13503us) write_op.cc:270] Start()
1211 20:55:13.633560 (+ 30us) write_op.cc:276] Timestamp: P: 1765486509256418 usec, L: 0
1211 20:55:13.658233 (+ 24673us) write_op.cc:270] Start()
1211 20:55:13.658261 (+ 28us) write_op.cc:276] Timestamp: P: 1765486509258791 usec, L: 0
1211 20:55:13.661740 (+ 3479us) write_op.cc:270] Start()
1211 20:55:13.661769 (+ 29us) write_op.cc:276] Timestamp: P: 1765486509293823 usec, L: 0
1211 20:55:13.665406 (+ 3637us) write_op.cc:270] Start()
1211 20:55:13.668511 (+ 3105us) write_op.cc:276] Timestamp: P: 1765486509306551 usec, L: 0
1211 20:55:13.670822 (+ 2311us) write_op.cc:270] Start()
1211 20:55:13.670846 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509322825 usec, L: 0
1211 20:55:13.674735 (+ 3889us) write_op.cc:270] Start()
1211 20:55:13.674762 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509343255 usec, L: 0
1211 20:55:13.674960 (+ 198us) write_op.cc:270] Start()
1211 20:55:13.674986 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509355437 usec, L: 0
1211 20:55:13.675216 (+ 230us) write_op.cc:270] Start()
1211 20:55:13.675242 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509368111 usec, L: 0
1211 20:55:13.695413 (+ 20171us) write_op.cc:270] Start()
1211 20:55:13.695435 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509384638 usec, L: 0
1211 20:55:13.701021 (+ 5586us) write_op.cc:270] Start()
1211 20:55:13.701049 (+ 28us) write_op.cc:276] Timestamp: P: 1765486509406848 usec, L: 0
1211 20:55:13.722466 (+ 21417us) write_op.cc:270] Start()
1211 20:55:13.722496 (+ 30us) write_op.cc:276] Timestamp: P: 1765486509432075 usec, L: 0
1211 20:55:13.748018 (+ 25522us) write_op.cc:270] Start()
1211 20:55:13.748049 (+ 31us) write_op.cc:276] Timestamp: P: 1765486509436697 usec, L: 0
1211 20:55:13.754235 (+ 6186us) write_op.cc:270] Start()
1211 20:55:13.754262 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509445176 usec, L: 0
1211 20:55:13.765738 (+ 11476us) write_op.cc:270] Start()
1211 20:55:13.765764 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509447987 usec, L: 0
1211 20:55:13.785460 (+ 19696us) write_op.cc:270] Start()
1211 20:55:13.785507 (+ 47us) write_op.cc:276] Timestamp: P: 1765486509471866 usec, L: 0
1211 20:55:13.792814 (+ 7307us) write_op.cc:270] Start()
1211 20:55:13.792841 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509486513 usec, L: 0
1211 20:55:13.793070 (+ 229us) write_op.cc:270] Start()
1211 20:55:13.793108 (+ 38us) write_op.cc:276] Timestamp: P: 1765486509505361 usec, L: 0
1211 20:55:13.800347 (+ 7239us) write_op.cc:270] Start()
1211 20:55:13.800372 (+ 25us) write_op.cc:276] Timestamp: P: 1765486509517610 usec, L: 0
1211 20:55:13.800579 (+ 207us) write_op.cc:270] Start()
1211 20:55:13.800601 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509526444 usec, L: 0
1211 20:55:13.800795 (+ 194us) write_op.cc:270] Start()
1211 20:55:13.827591 (+ 26796us) write_op.cc:276] Timestamp: P: 1765486509542135 usec, L: 0
1211 20:55:13.827830 (+ 239us) write_op.cc:270] Start()
1211 20:55:13.827860 (+ 30us) write_op.cc:276] Timestamp: P: 1765486509567894 usec, L: 0
1211 20:55:13.828047 (+ 187us) write_op.cc:270] Start()
1211 20:55:13.828068 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509585826 usec, L: 0
1211 20:55:13.828266 (+ 198us) write_op.cc:270] Start()
1211 20:55:13.828286 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509587735 usec, L: 0
1211 20:55:13.864228 (+ 35942us) write_op.cc:270] Start()
1211 20:55:13.864255 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509606080 usec, L: 0
1211 20:55:13.864429 (+ 174us) write_op.cc:270] Start()
1211 20:55:13.864449 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509618436 usec, L: 0
1211 20:55:13.864593 (+ 144us) write_op.cc:270] Start()
1211 20:55:13.864612 (+ 19us) write_op.cc:276] Timestamp: P: 1765486509656871 usec, L: 0
1211 20:55:13.864787 (+ 175us) write_op.cc:270] Start()
1211 20:55:13.864805 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509658618 usec, L: 0
1211 20:55:13.864931 (+ 126us) write_op.cc:270] Start()
1211 20:55:13.864966 (+ 35us) write_op.cc:276] Timestamp: P: 1765486509684428 usec, L: 0
1211 20:55:13.876500 (+ 11534us) write_op.cc:270] Start()
1211 20:55:13.876528 (+ 28us) write_op.cc:276] Timestamp: P: 1765486509704863 usec, L: 0
1211 20:55:13.876705 (+ 177us) write_op.cc:270] Start()
1211 20:55:13.876728 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509729786 usec, L: 0
1211 20:55:13.879413 (+ 2685us) write_op.cc:270] Start()
1211 20:55:13.879444 (+ 31us) write_op.cc:276] Timestamp: P: 1765486509732530 usec, L: 0
1211 20:55:13.897055 (+ 17611us) write_op.cc:270] Start()
1211 20:55:13.914123 (+ 17068us) write_op.cc:276] Timestamp: P: 1765486509776431 usec, L: 0
1211 20:55:13.915549 (+ 1426us) write_op.cc:270] Start()
1211 20:55:13.915569 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509792050 usec, L: 0
1211 20:55:13.917732 (+ 2163us) write_op.cc:270] Start()
1211 20:55:13.917754 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509823088 usec, L: 0
1211 20:55:13.919342 (+ 1588us) write_op.cc:270] Start()
1211 20:55:13.919373 (+ 31us) write_op.cc:276] Timestamp: P: 1765486509824059 usec, L: 0
1211 20:55:13.919523 (+ 150us) write_op.cc:270] Start()
1211 20:55:13.929361 (+ 9838us) write_op.cc:276] Timestamp: P: 1765486509833773 usec, L: 0
1211 20:55:13.939687 (+ 10326us) write_op.cc:270] Start()
1211 20:55:13.939729 (+ 42us) write_op.cc:276] Timestamp: P: 1765486509852729 usec, L: 0
1211 20:55:13.939913 (+ 184us) write_op.cc:270] Start()
1211 20:55:13.939930 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509896178 usec, L: 0
1211 20:55:13.940052 (+ 122us) write_op.cc:270] Start()
1211 20:55:13.940068 (+ 16us) write_op.cc:276] Timestamp: P: 1765486509910949 usec, L: 0
1211 20:55:13.941921 (+ 1853us) write_op.cc:270] Start()
1211 20:55:13.941942 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509920924 usec, L: 0
1211 20:55:13.942109 (+ 167us) write_op.cc:270] Start()
1211 20:55:13.942127 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509931227 usec, L: 0
1211 20:55:13.942237 (+ 110us) write_op.cc:270] Start()
1211 20:55:13.942254 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509973575 usec, L: 0
1211 20:55:13.943557 (+ 1303us) write_op.cc:270] Start()
1211 20:55:13.943579 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509979943 usec, L: 0
1211 20:55:13.943741 (+ 162us) write_op.cc:270] Start()
1211 20:55:13.943764 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509981003 usec, L: 0
1211 20:55:13.943909 (+ 145us) write_op.cc:270] Start()
1211 20:55:13.943926 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509986977 usec, L: 0
1211 20:55:13.944032 (+ 106us) write_op.cc:270] Start()
1211 20:55:13.944049 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510033819 usec, L: 0
1211 20:55:13.944183 (+ 134us) write_op.cc:270] Start()
1211 20:55:13.944199 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510038807 usec, L: 0
1211 20:55:13.944325 (+ 126us) write_op.cc:270] Start()
1211 20:55:13.944341 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510067995 usec, L: 0
1211 20:55:13.944463 (+ 122us) write_op.cc:270] Start()
1211 20:55:13.944479 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510090945 usec, L: 0
1211 20:55:13.944583 (+ 104us) write_op.cc:270] Start()
1211 20:55:13.944596 (+ 13us) write_op.cc:276] Timestamp: P: 1765486510107722 usec, L: 0
1211 20:55:13.944690 (+ 94us) write_op.cc:270] Start()
1211 20:55:13.944703 (+ 13us) write_op.cc:276] Timestamp: P: 1765486510113087 usec, L: 0
1211 20:55:13.944803 (+ 100us) write_op.cc:270] Start()
1211 20:55:13.944818 (+ 15us) write_op.cc:276] Timestamp: P: 1765486510129735 usec, L: 0
1211 20:55:13.944909 (+ 91us) write_op.cc:270] Start()
1211 20:55:13.944923 (+ 14us) write_op.cc:276] Timestamp: P: 1765486510191029 usec, L: 0
1211 20:55:13.945010 (+ 87us) write_op.cc:270] Start()
1211 20:55:13.945023 (+ 13us) write_op.cc:276] Timestamp: P: 1765486510205910 usec, L: 0
1211 20:55:13.945142 (+ 119us) write_op.cc:270] Start()
1211 20:55:13.945157 (+ 15us) write_op.cc:276] Timestamp: P: 1765486510228547 usec, L: 0
1211 20:55:13.945267 (+ 110us) write_op.cc:270] Start()
1211 20:55:13.945282 (+ 15us) write_op.cc:276] Timestamp: P: 1765486510242372 usec, L: 0
1211 20:55:13.945386 (+ 104us) write_op.cc:270] Start()
1211 20:55:13.945400 (+ 14us) write_op.cc:276] Timestamp: P: 1765486510280267 usec, L: 0
1211 20:55:13.945510 (+ 110us) write_op.cc:270] Start()
1211 20:55:13.945524 (+ 14us) write_op.cc:276] Timestamp: P: 1765486510311184 usec, L: 0
1211 20:55:13.945646 (+ 122us) write_op.cc:270] Start()
1211 20:55:13.945659 (+ 13us) write_op.cc:276] Timestamp: P: 1765486510329345 usec, L: 0
1211 20:55:13.945760 (+ 101us) write_op.cc:270] Start()
1211 20:55:13.945775 (+ 15us) write_op.cc:276] Timestamp: P: 1765486510362692 usec, L: 0
1211 20:55:13.945878 (+ 103us) write_op.cc:270] Start()
1211 20:55:13.945892 (+ 14us) write_op.cc:276] Timestamp: P: 1765486510389148 usec, L: 0
1211 20:55:13.945984 (+ 92us) write_op.cc:270] Start()
1211 20:55:13.951751 (+ 5767us) write_op.cc:276] Timestamp: P: 1765486510398828 usec, L: 0
1211 20:55:13.965066 (+ 13315us) write_op.cc:270] Start()
1211 20:55:13.965091 (+ 25us) write_op.cc:276] Timestamp: P: 1765486510421696 usec, L: 0
1211 20:55:13.978060 (+ 12969us) write_op.cc:270] Start()
1211 20:55:13.978083 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510443847 usec, L: 0
1211 20:55:13.978465 (+ 382us) write_op.cc:270] Start()
1211 20:55:13.978486 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510485706 usec, L: 0
1211 20:55:13.978641 (+ 155us) write_op.cc:270] Start()
1211 20:55:13.978657 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510486267 usec, L: 0
1211 20:55:13.978831 (+ 174us) write_op.cc:270] Start()
1211 20:55:13.978850 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510487899 usec, L: 0
1211 20:55:13.979022 (+ 172us) write_op.cc:270] Start()
1211 20:55:13.979049 (+ 27us) write_op.cc:276] Timestamp: P: 1765486510517896 usec, L: 0
1211 20:55:14.006159 (+ 27110us) write_op.cc:270] Start()
1211 20:55:14.006182 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510552727 usec, L: 0
1211 20:55:14.006421 (+ 239us) write_op.cc:270] Start()
1211 20:55:14.006487 (+ 66us) write_op.cc:276] Timestamp: P: 1765486510563882 usec, L: 0
1211 20:55:14.006684 (+ 197us) write_op.cc:270] Start()
1211 20:55:14.006703 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510564857 usec, L: 0
1211 20:55:14.006925 (+ 222us) write_op.cc:270] Start()
1211 20:55:14.006945 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510568802 usec, L: 0
1211 20:55:14.007178 (+ 233us) write_op.cc:270] Start()
1211 20:55:14.007198 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510610629 usec, L: 0
1211 20:55:14.007377 (+ 179us) write_op.cc:270] Start()
1211 20:55:14.007397 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510614540 usec, L: 0
1211 20:55:14.007582 (+ 185us) write_op.cc:270] Start()
1211 20:55:14.007601 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510664157 usec, L: 0
1211 20:55:14.007776 (+ 175us) write_op.cc:270] Start()
1211 20:55:14.007795 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510683879 usec, L: 0
1211 20:55:14.007968 (+ 173us) write_op.cc:270] Start()
1211 20:55:14.007986 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510706981 usec, L: 0
1211 20:55:14.008141 (+ 155us) write_op.cc:270] Start()
1211 20:55:14.008157 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510713809 usec, L: 0
1211 20:55:14.008327 (+ 170us) write_op.cc:270] Start()
1211 20:55:14.008344 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510743985 usec, L: 0
1211 20:55:14.008515 (+ 171us) write_op.cc:270] Start()
1211 20:55:14.008532 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510746364 usec, L: 0
1211 20:55:14.008673 (+ 141us) write_op.cc:270] Start()
1211 20:55:14.008690 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510748373 usec, L: 0
1211 20:55:14.008833 (+ 143us) write_op.cc:270] Start()
1211 20:55:14.008849 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510779759 usec, L: 0
1211 20:55:14.009014 (+ 165us) write_op.cc:270] Start()
1211 20:55:14.009033 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510805527 usec, L: 0
1211 20:55:14.009208 (+ 175us) write_op.cc:270] Start()
1211 20:55:14.009240 (+ 32us) write_op.cc:276] Timestamp: P: 1765486510814994 usec, L: 0
1211 20:55:14.009429 (+ 189us) write_op.cc:270] Start()
1211 20:55:14.009444 (+ 15us) write_op.cc:276] Timestamp: P: 1765486510817331 usec, L: 0
1211 20:55:14.009595 (+ 151us) write_op.cc:270] Start()
1211 20:55:14.009611 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510817854 usec, L: 0
1211 20:55:14.009746 (+ 135us) write_op.cc:270] Start()
1211 20:55:14.009759 (+ 13us) write_op.cc:276] Timestamp: P: 1765486510842597 usec, L: 0
1211 20:55:14.011171 (+ 1412us) write_op.cc:270] Start()
1211 20:55:14.011193 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510893265 usec, L: 0
1211 20:55:14.014123 (+ 2930us) write_op.cc:270] Start()
1211 20:55:14.014144 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510944575 usec, L: 0
1211 20:55:14.014374 (+ 230us) write_op.cc:270] Start()
1211 20:55:14.014392 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510997798 usec, L: 0
1211 20:55:14.018118 (+ 3726us) write_op.cc:270] Start()
1211 20:55:14.018139 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510999518 usec, L: 0
1211 20:55:14.066177 (+ 48038us) write_op.cc:270] Start()
1211 20:55:14.066198 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511013834 usec, L: 0
1211 20:55:14.066362 (+ 164us) write_op.cc:270] Start()
1211 20:55:14.066382 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511047146 usec, L: 0
1211 20:55:14.066566 (+ 184us) write_op.cc:270] Start()
1211 20:55:14.066587 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511073505 usec, L: 0
1211 20:55:14.066710 (+ 123us) write_op.cc:270] Start()
1211 20:55:14.066727 (+ 17us) write_op.cc:276] Timestamp: P: 1765486511092324 usec, L: 0
1211 20:55:14.066853 (+ 126us) write_op.cc:270] Start()
1211 20:55:14.066871 (+ 18us) write_op.cc:276] Timestamp: P: 1765486511115980 usec, L: 0
1211 20:55:14.067114 (+ 243us) write_op.cc:270] Start()
1211 20:55:14.067135 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511116649 usec, L: 0
1211 20:55:14.067272 (+ 137us) write_op.cc:270] Start()
1211 20:55:14.067291 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511119243 usec, L: 0
1211 20:55:14.067437 (+ 146us) write_op.cc:270] Start()
1211 20:55:14.067456 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511153704 usec, L: 0
1211 20:55:14.067882 (+ 426us) write_op.cc:270] Start()
1211 20:55:14.067927 (+ 45us) write_op.cc:276] Timestamp: P: 1765486511198628 usec, L: 0
1211 20:55:14.068137 (+ 210us) write_op.cc:270] Start()
1211 20:55:14.068157 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511219924 usec, L: 0
1211 20:55:14.068373 (+ 216us) write_op.cc:270] Start()
1211 20:55:14.068392 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511223903 usec, L: 0
1211 20:55:14.068516 (+ 124us) write_op.cc:270] Start()
1211 20:55:14.068557 (+ 41us) write_op.cc:276] Timestamp: P: 1765486511228930 usec, L: 0
1211 20:55:14.068686 (+ 129us) write_op.cc:270] Start()
1211 20:55:14.068703 (+ 17us) write_op.cc:276] Timestamp: P: 1765486511285738 usec, L: 0
1211 20:55:14.068828 (+ 125us) write_op.cc:270] Start()
1211 20:55:14.068848 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511286466 usec, L: 0
1211 20:55:14.068984 (+ 136us) write_op.cc:270] Start()
1211 20:55:14.069003 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511289342 usec, L: 0
1211 20:55:14.069178 (+ 175us) write_op.cc:270] Start()
1211 20:55:14.069198 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511325955 usec, L: 0
1211 20:55:14.069378 (+ 180us) write_op.cc:270] Start()
1211 20:55:14.069397 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511351818 usec, L: 0
1211 20:55:14.082726 (+ 13329us) write_op.cc:270] Start()
1211 20:55:14.082754 (+ 28us) write_op.cc:276] Timestamp: P: 1765486511362877 usec, L: 0
1211 20:55:14.082955 (+ 201us) write_op.cc:270] Start()
1211 20:55:14.082979 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511396742 usec, L: 0
1211 20:55:14.083160 (+ 181us) write_op.cc:270] Start()
1211 20:55:14.083182 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511399594 usec, L: 0
1211 20:55:14.083351 (+ 169us) write_op.cc:270] Start()
1211 20:55:14.083383 (+ 32us) write_op.cc:276] Timestamp: P: 1765486511401349 usec, L: 0
1211 20:55:14.083551 (+ 168us) write_op.cc:270] Start()
1211 20:55:14.083574 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511462768 usec, L: 0
1211 20:55:14.085806 (+ 2232us) write_op.cc:270] Start()
1211 20:55:14.085830 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511483354 usec, L: 0
1211 20:55:14.086038 (+ 208us) write_op.cc:270] Start()
1211 20:55:14.086062 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511488680 usec, L: 0
1211 20:55:14.086315 (+ 253us) write_op.cc:270] Start()
1211 20:55:14.086341 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511501039 usec, L: 0
1211 20:55:14.086530 (+ 189us) write_op.cc:270] Start()
1211 20:55:14.086551 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511533079 usec, L: 0
1211 20:55:14.086695 (+ 144us) write_op.cc:270] Start()
1211 20:55:14.086716 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511547534 usec, L: 0
1211 20:55:14.086894 (+ 178us) write_op.cc:270] Start()
1211 20:55:14.086919 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511572359 usec, L: 0
1211 20:55:14.087193 (+ 274us) write_op.cc:270] Start()
1211 20:55:14.087214 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511578722 usec, L: 0
1211 20:55:14.087420 (+ 206us) write_op.cc:270] Start()
1211 20:55:14.087443 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511589899 usec, L: 0
1211 20:55:14.087656 (+ 213us) write_op.cc:270] Start()
1211 20:55:14.087678 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511623035 usec, L: 0
1211 20:55:14.087886 (+ 208us) write_op.cc:270] Start()
1211 20:55:14.087911 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511644586 usec, L: 0
1211 20:55:14.088096 (+ 185us) write_op.cc:270] Start()
1211 20:55:14.088117 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511652541 usec, L: 0
1211 20:55:14.091107 (+ 2990us) write_op.cc:270] Start()
1211 20:55:14.091131 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511668833 usec, L: 0
1211 20:55:14.092107 (+ 976us) write_op.cc:270] Start()
1211 20:55:14.092130 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511669488 usec, L: 0
1211 20:55:14.092321 (+ 191us) write_op.cc:270] Start()
1211 20:55:14.092342 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511714802 usec, L: 0
1211 20:55:14.110413 (+ 18071us) write_op.cc:270] Start()
1211 20:55:14.110440 (+ 27us) write_op.cc:276] Timestamp: P: 1765486511743886 usec, L: 0
1211 20:55:14.110673 (+ 233us) write_op.cc:270] Start()
1211 20:55:14.110693 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511758667 usec, L: 0
1211 20:55:14.117852 (+ 7159us) write_op.cc:270] Start()
1211 20:55:14.117875 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511759738 usec, L: 0
1211 20:55:14.137976 (+ 20101us) write_op.cc:270] Start()
1211 20:55:14.137996 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511766269 usec, L: 0
1211 20:55:14.139868 (+ 1872us) write_op.cc:270] Start()
1211 20:55:14.139894 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511802265 usec, L: 0
1211 20:55:14.140082 (+ 188us) write_op.cc:270] Start()
1211 20:55:14.140103 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511821124 usec, L: 0
1211 20:55:14.140294 (+ 191us) write_op.cc:270] Start()
1211 20:55:14.140313 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511832040 usec, L: 0
1211 20:55:14.140498 (+ 185us) write_op.cc:270] Start()
1211 20:55:14.140519 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511840199 usec, L: 0
1211 20:55:14.140710 (+ 191us) write_op.cc:270] Start()
1211 20:55:14.140731 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511900709 usec, L: 0
1211 20:55:14.140937 (+ 206us) write_op.cc:270] Start()
1211 20:55:14.140966 (+ 29us) write_op.cc:276] Timestamp: P: 1765486511920184 usec, L: 0
1211 20:55:14.141132 (+ 166us) write_op.cc:270] Start()
1211 20:55:14.141151 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511921058 usec, L: 0
1211 20:55:14.141370 (+ 219us) write_op.cc:270] Start()
1211 20:55:14.141390 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511925617 usec, L: 0
1211 20:55:14.141564 (+ 174us) write_op.cc:270] Start()
1211 20:55:14.141583 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511957435 usec, L: 0
1211 20:55:14.164540 (+ 22957us) write_op.cc:270] Start()
1211 20:55:14.164566 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511976626 usec, L: 0
1211 20:55:14.164800 (+ 234us) write_op.cc:270] Start()
1211 20:55:14.164822 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511990730 usec, L: 0
1211 20:55:14.165116 (+ 294us) write_op.cc:270] Start()
1211 20:55:14.165136 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512004549 usec, L: 0
1211 20:55:14.165291 (+ 155us) write_op.cc:270] Start()
1211 20:55:14.165309 (+ 18us) write_op.cc:276] Timestamp: P: 1765486512018967 usec, L: 0
1211 20:55:14.165539 (+ 230us) write_op.cc:270] Start()
1211 20:55:14.165561 (+ 22us) write_op.cc:276] Timestamp: P: 1765486512048587 usec, L: 0
1211 20:55:14.165762 (+ 201us) write_op.cc:270] Start()
1211 20:55:14.165785 (+ 23us) write_op.cc:276] Timestamp: P: 1765486512059802 usec, L: 0
1211 20:55:14.165972 (+ 187us) write_op.cc:270] Start()
1211 20:55:14.165994 (+ 22us) write_op.cc:276] Timestamp: P: 1765486512063930 usec, L: 0
1211 20:55:14.166249 (+ 255us) write_op.cc:270] Start()
1211 20:55:14.166271 (+ 22us) write_op.cc:276] Timestamp: P: 1765486512106823 usec, L: 0
1211 20:55:14.166470 (+ 199us) write_op.cc:270] Start()
1211 20:55:14.166493 (+ 23us) write_op.cc:276] Timestamp: P: 1765486512108565 usec, L: 0
1211 20:55:14.166671 (+ 178us) write_op.cc:270] Start()
1211 20:55:14.166691 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512112172 usec, L: 0
1211 20:55:14.166850 (+ 159us) write_op.cc:270] Start()
1211 20:55:14.166869 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512112817 usec, L: 0
1211 20:55:14.167088 (+ 219us) write_op.cc:270] Start()
1211 20:55:14.167109 (+ 21us) write_op.cc:276] Timestamp: P: 1765486512156260 usec, L: 0
1211 20:55:14.167300 (+ 191us) write_op.cc:270] Start()
1211 20:55:14.167320 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512180883 usec, L: 0
1211 20:55:14.167560 (+ 240us) write_op.cc:270] Start()
1211 20:55:14.167582 (+ 22us) write_op.cc:276] Timestamp: P: 1765486512181560 usec, L: 0
1211 20:55:14.167769 (+ 187us) write_op.cc:270] Start()
1211 20:55:14.167790 (+ 21us) write_op.cc:276] Timestamp: P: 1765486512189084 usec, L: 0
1211 20:55:14.167963 (+ 173us) write_op.cc:270] Start()
1211 20:55:14.167981 (+ 18us) write_op.cc:276] Timestamp: P: 1765486512224837 usec, L: 0
1211 20:55:14.168159 (+ 178us) write_op.cc:270] Start()
1211 20:55:14.168177 (+ 18us) write_op.cc:276] Timestamp: P: 1765486512246059 usec, L: 0
1211 20:55:14.168376 (+ 199us) write_op.cc:270] Start()
1211 20:55:14.168396 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512252722 usec, L: 0
1211 20:55:14.168578 (+ 182us) write_op.cc:270] Start()
1211 20:55:14.168596 (+ 18us) write_op.cc:276] Timestamp: P: 1765486512270273 usec, L: 0
1211 20:55:14.180618 (+ 12022us) write_op.cc:270] Start()
1211 20:55:14.180654 (+ 36us) write_op.cc:276] Timestamp: P: 1765486512293323 usec, L: 0
1211 20:55:14.180836 (+ 182us) write_op.cc:270] Start()
1211 20:55:14.180855 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512297392 usec, L: 0
1211 20:55:14.189614 (+ 8759us) write_op.cc:270] Start()
1211 20:55:14.189634 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512325913 usec, L: 0
1211 20:55:14.189803 (+ 169us) write_op.cc:270] Start()
1211 20:55:14.189822 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512346312 usec, L: 0
1211 20:55:14.190245 (+ 423us) write_op.cc:270] Start()
1211 20:55:14.190282 (+ 37us) write_op.cc:276] Timestamp: P: 1765486512349104 usec, L: 0
1211 20:55:14.190494 (+ 212us) write_op.cc:270] Start()
1211 20:55:14.190514 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512361306 usec, L: 0
1211 20:55:14.190847 (+ 333us) write_op.cc:270] Start()
1211 20:55:14.190873 (+ 26us) write_op.cc:276] Timestamp: P: 1765486512387947 usec, L: 0
1211 20:55:14.191044 (+ 171us) write_op.cc:270] Start()
1211 20:55:14.191064 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512397619 usec, L: 0
1211 20:55:14.191336 (+ 272us) write_op.cc:270] Start()
1211 20:55:14.191356 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512401823 usec, L: 0
1211 20:55:14.191668 (+ 312us) write_op.cc:270] Start()
1211 20:55:14.191688 (+ 20us) write_op.cc:276] Timestamp: P: 17654865
I20251211 20:55:14.735421 7035 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:14.736212 7035 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:14.739261 6945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.195:38469 (request call id 10) took 4518 ms. Trace:
I20251211 20:55:14.739722 7057 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:14.743752 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 986 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:14.744496 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 986 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:14.751443 7035 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:14.757175 6872 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:14.797597 7035 raft_consensus.cc:2749] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:14.739881 7057 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:14.811693 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:14.812971 7057 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:14.858239 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 884 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:14.863256 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:14.863675 7057 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:10.958237 6946 rpcz_store.cc:276] 1211 20:55:08.366170 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:08.373138 (+ 6968us) service_pool.cc:225] Handling call
1211 20:55:08.559788 (+186650us) raft_consensus.cc:1471] Updating replica for 724 ops
1211 20:55:08.687182 (+127394us) raft_consensus.cc:1534] Early marking committed up to index 181
1211 20:55:08.757911 (+ 70729us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:55:08.757918 (+ 7us) write_op.cc:454] Released schema lock
1211 20:55:08.774073 (+ 16155us) write_op.cc:489] Releasing partition, row and schema locks
1211 20:55:08.774087 (+ 14us) write_op.cc:454] Released schema lock
1211 20:55:08.776726 (+ 2639us) raft_consensus.cc:1539] Triggering prepare for 703 ops
1211 20:55:08.776862 (+ 136us) write_op.cc:270] Start()
1211 20:55:08.776889 (+ 27us) write_op.cc:276] Timestamp: P: 1765486495835864 usec, L: 0
1211 20:55:08.791026 (+ 14137us) write_op.cc:270] Start()
1211 20:55:08.791056 (+ 30us) write_op.cc:276] Timestamp: P: 1765486495867621 usec, L: 0
1211 20:55:08.791261 (+ 205us) write_op.cc:270] Start()
1211 20:55:08.791288 (+ 27us) write_op.cc:276] Timestamp: P: 1765486495882522 usec, L: 0
1211 20:55:08.791440 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.791458 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495898521 usec, L: 0
1211 20:55:08.791677 (+ 219us) write_op.cc:270] Start()
1211 20:55:08.791697 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495903456 usec, L: 0
1211 20:55:08.791874 (+ 177us) write_op.cc:270] Start()
1211 20:55:08.791894 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495912173 usec, L: 0
1211 20:55:08.792030 (+ 136us) write_op.cc:270] Start()
1211 20:55:08.792048 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495933607 usec, L: 0
1211 20:55:08.792232 (+ 184us) write_op.cc:270] Start()
1211 20:55:08.792254 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495949650 usec, L: 0
1211 20:55:08.792392 (+ 138us) write_op.cc:270] Start()
1211 20:55:08.792411 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495959731 usec, L: 0
1211 20:55:08.792590 (+ 179us) write_op.cc:270] Start()
1211 20:55:08.792610 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495961805 usec, L: 0
1211 20:55:08.792757 (+ 147us) write_op.cc:270] Start()
1211 20:55:08.792776 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495973581 usec, L: 0
1211 20:55:08.792931 (+ 155us) write_op.cc:270] Start()
1211 20:55:08.792952 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495999339 usec, L: 0
1211 20:55:08.793120 (+ 168us) write_op.cc:270] Start()
1211 20:55:08.793141 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496007472 usec, L: 0
1211 20:55:08.793316 (+ 175us) write_op.cc:270] Start()
1211 20:55:08.793338 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496017765 usec, L: 0
1211 20:55:08.793492 (+ 154us) write_op.cc:270] Start()
1211 20:55:08.793511 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496034123 usec, L: 0
1211 20:55:08.793639 (+ 128us) write_op.cc:270] Start()
1211 20:55:08.793657 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496058339 usec, L: 0
1211 20:55:08.793789 (+ 132us) write_op.cc:270] Start()
1211 20:55:08.793804 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496061174 usec, L: 0
1211 20:55:08.798631 (+ 4827us) write_op.cc:270] Start()
1211 20:55:08.798653 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496074424 usec, L: 0
1211 20:55:08.803600 (+ 4947us) write_op.cc:270] Start()
1211 20:55:08.803624 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496080735 usec, L: 0
1211 20:55:08.807760 (+ 4136us) write_op.cc:270] Start()
1211 20:55:08.813851 (+ 6091us) write_op.cc:276] Timestamp: P: 1765486496098526 usec, L: 0
1211 20:55:08.814522 (+ 671us) write_op.cc:270] Start()
1211 20:55:08.814544 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496116592 usec, L: 0
1211 20:55:08.823069 (+ 8525us) write_op.cc:270] Start()
1211 20:55:08.823367 (+ 298us) write_op.cc:276] Timestamp: P: 1765486496122720 usec, L: 0
1211 20:55:08.823608 (+ 241us) write_op.cc:270] Start()
1211 20:55:08.823627 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496143794 usec, L: 0
1211 20:55:08.824241 (+ 614us) write_op.cc:270] Start()
1211 20:55:08.824262 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496148112 usec, L: 0
1211 20:55:08.824855 (+ 593us) write_op.cc:270] Start()
1211 20:55:08.824874 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496159805 usec, L: 0
1211 20:55:08.825141 (+ 267us) write_op.cc:270] Start()
1211 20:55:08.825160 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496163482 usec, L: 0
1211 20:55:08.825308 (+ 148us) write_op.cc:270] Start()
1211 20:55:08.825324 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496187855 usec, L: 0
1211 20:55:08.825432 (+ 108us) write_op.cc:270] Start()
1211 20:55:08.825447 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496191359 usec, L: 0
1211 20:55:08.825567 (+ 120us) write_op.cc:270] Start()
1211 20:55:08.825583 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496210253 usec, L: 0
1211 20:55:08.825706 (+ 123us) write_op.cc:270] Start()
1211 20:55:08.825721 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496221179 usec, L: 0
1211 20:55:08.825849 (+ 128us) write_op.cc:270] Start()
1211 20:55:08.825868 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496236751 usec, L: 0
1211 20:55:08.826017 (+ 149us) write_op.cc:270] Start()
1211 20:55:08.826034 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496251121 usec, L: 0
1211 20:55:08.826420 (+ 386us) write_op.cc:270] Start()
1211 20:55:08.826435 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496263839 usec, L: 0
1211 20:55:08.826563 (+ 128us) write_op.cc:270] Start()
1211 20:55:08.826579 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496285587 usec, L: 0
1211 20:55:08.826695 (+ 116us) write_op.cc:270] Start()
1211 20:55:08.826715 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496289340 usec, L: 0
1211 20:55:08.826825 (+ 110us) write_op.cc:270] Start()
1211 20:55:08.826840 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496297660 usec, L: 0
1211 20:55:08.826989 (+ 149us) write_op.cc:270] Start()
1211 20:55:08.834626 (+ 7637us) write_op.cc:276] Timestamp: P: 1765486496317033 usec, L: 0
1211 20:55:08.835557 (+ 931us) write_op.cc:270] Start()
1211 20:55:08.835577 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496338159 usec, L: 0
1211 20:55:08.835849 (+ 272us) write_op.cc:270] Start()
1211 20:55:08.835872 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496339929 usec, L: 0
1211 20:55:08.836033 (+ 161us) write_op.cc:270] Start()
1211 20:55:08.836050 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496354744 usec, L: 0
1211 20:55:08.836182 (+ 132us) write_op.cc:270] Start()
1211 20:55:08.836198 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496379704 usec, L: 0
1211 20:55:08.836305 (+ 107us) write_op.cc:270] Start()
1211 20:55:08.836319 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496391455 usec, L: 0
1211 20:55:08.836425 (+ 106us) write_op.cc:270] Start()
1211 20:55:08.836440 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496399228 usec, L: 0
1211 20:55:08.836545 (+ 105us) write_op.cc:270] Start()
1211 20:55:08.836559 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496408932 usec, L: 0
1211 20:55:08.836681 (+ 122us) write_op.cc:270] Start()
1211 20:55:08.836697 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496417416 usec, L: 0
1211 20:55:08.836810 (+ 113us) write_op.cc:270] Start()
1211 20:55:08.836824 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496435258 usec, L: 0
1211 20:55:08.836929 (+ 105us) write_op.cc:270] Start()
1211 20:55:08.836943 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496442932 usec, L: 0
1211 20:55:08.837085 (+ 142us) write_op.cc:270] Start()
1211 20:55:08.837101 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496469563 usec, L: 0
1211 20:55:08.837213 (+ 112us) write_op.cc:270] Start()
1211 20:55:08.837229 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496489068 usec, L: 0
1211 20:55:08.837334 (+ 105us) write_op.cc:270] Start()
1211 20:55:08.837349 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496495209 usec, L: 0
1211 20:55:08.837528 (+ 179us) write_op.cc:270] Start()
1211 20:55:08.837556 (+ 28us) write_op.cc:276] Timestamp: P: 1765486496495672 usec, L: 0
1211 20:55:08.837654 (+ 98us) write_op.cc:270] Start()
1211 20:55:08.837668 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496510469 usec, L: 0
1211 20:55:08.837772 (+ 104us) write_op.cc:270] Start()
1211 20:55:08.837787 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496527883 usec, L: 0
1211 20:55:08.837895 (+ 108us) write_op.cc:270] Start()
1211 20:55:08.837908 (+ 13us) write_op.cc:276] Timestamp: P: 1765486496533305 usec, L: 0
1211 20:55:08.838029 (+ 121us) write_op.cc:270] Start()
1211 20:55:08.838046 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496541742 usec, L: 0
1211 20:55:08.838175 (+ 129us) write_op.cc:270] Start()
1211 20:55:08.838189 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496558223 usec, L: 0
1211 20:55:08.838283 (+ 94us) write_op.cc:270] Start()
1211 20:55:08.838297 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496572380 usec, L: 0
1211 20:55:08.838410 (+ 113us) write_op.cc:270] Start()
1211 20:55:08.838426 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496578440 usec, L: 0
1211 20:55:08.838537 (+ 111us) write_op.cc:270] Start()
1211 20:55:08.838552 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496579015 usec, L: 0
1211 20:55:08.838650 (+ 98us) write_op.cc:270] Start()
1211 20:55:08.838664 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496608299 usec, L: 0
1211 20:55:08.838781 (+ 117us) write_op.cc:270] Start()
1211 20:55:08.838794 (+ 13us) write_op.cc:276] Timestamp: P: 1765486496625426 usec, L: 0
1211 20:55:08.838898 (+ 104us) write_op.cc:270] Start()
1211 20:55:08.838912 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496628980 usec, L: 0
1211 20:55:08.839002 (+ 90us) write_op.cc:270] Start()
1211 20:55:08.839016 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496635858 usec, L: 0
1211 20:55:08.839135 (+ 119us) write_op.cc:270] Start()
1211 20:55:08.839149 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496645357 usec, L: 0
1211 20:55:08.839291 (+ 142us) write_op.cc:270] Start()
1211 20:55:08.839307 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496671092 usec, L: 0
1211 20:55:08.839439 (+ 132us) write_op.cc:270] Start()
1211 20:55:08.839454 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496690460 usec, L: 0
1211 20:55:08.839575 (+ 121us) write_op.cc:270] Start()
1211 20:55:08.839591 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496693834 usec, L: 0
1211 20:55:08.839698 (+ 107us) write_op.cc:270] Start()
1211 20:55:08.839713 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496699333 usec, L: 0
1211 20:55:08.839811 (+ 98us) write_op.cc:270] Start()
1211 20:55:08.839825 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496720327 usec, L: 0
1211 20:55:08.839919 (+ 94us) write_op.cc:270] Start()
1211 20:55:08.839933 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496738870 usec, L: 0
1211 20:55:08.853727 (+ 13794us) write_op.cc:270] Start()
1211 20:55:08.853748 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496748182 usec, L: 0
1211 20:55:08.879613 (+ 25865us) write_op.cc:270] Start()
1211 20:55:08.879639 (+ 26us) write_op.cc:276] Timestamp: P: 1765486496759072 usec, L: 0
1211 20:55:08.879812 (+ 173us) write_op.cc:270] Start()
1211 20:55:08.879832 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496781465 usec, L: 0
1211 20:55:08.879965 (+ 133us) write_op.cc:270] Start()
1211 20:55:08.879984 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496790480 usec, L: 0
1211 20:55:08.880156 (+ 172us) write_op.cc:270] Start()
1211 20:55:08.880177 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496813165 usec, L: 0
1211 20:55:08.880316 (+ 139us) write_op.cc:270] Start()
1211 20:55:08.880346 (+ 30us) write_op.cc:276] Timestamp: P: 1765486496815742 usec, L: 0
1211 20:55:08.880492 (+ 146us) write_op.cc:270] Start()
1211 20:55:08.880510 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496817109 usec, L: 0
1211 20:55:08.880695 (+ 185us) write_op.cc:270] Start()
1211 20:55:08.880717 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496854377 usec, L: 0
1211 20:55:08.880862 (+ 145us) write_op.cc:270] Start()
1211 20:55:08.880906 (+ 44us) write_op.cc:276] Timestamp: P: 1765486496867896 usec, L: 0
1211 20:55:08.881068 (+ 162us) write_op.cc:270] Start()
1211 20:55:08.881090 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496877252 usec, L: 0
1211 20:55:08.902730 (+ 21640us) write_op.cc:270] Start()
1211 20:55:08.902759 (+ 29us) write_op.cc:276] Timestamp: P: 1765486496894289 usec, L: 0
1211 20:55:08.902975 (+ 216us) write_op.cc:270] Start()
1211 20:55:08.902998 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496920751 usec, L: 0
1211 20:55:08.903169 (+ 171us) write_op.cc:270] Start()
1211 20:55:08.903189 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496932395 usec, L: 0
1211 20:55:08.906156 (+ 2967us) write_op.cc:270] Start()
1211 20:55:08.906222 (+ 66us) write_op.cc:276] Timestamp: P: 1765486496939264 usec, L: 0
1211 20:55:08.912656 (+ 6434us) write_op.cc:270] Start()
1211 20:55:08.912683 (+ 27us) write_op.cc:276] Timestamp: P: 1765486496959567 usec, L: 0
1211 20:55:08.914542 (+ 1859us) write_op.cc:270] Start()
1211 20:55:08.914562 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496976316 usec, L: 0
1211 20:55:08.914687 (+ 125us) write_op.cc:270] Start()
1211 20:55:08.914703 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496994754 usec, L: 0
1211 20:55:08.925567 (+ 10864us) write_op.cc:270] Start()
1211 20:55:08.925594 (+ 27us) write_op.cc:276] Timestamp: P: 1765486496997714 usec, L: 0
1211 20:55:08.928315 (+ 2721us) write_op.cc:270] Start()
1211 20:55:08.928329 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496999242 usec, L: 0
1211 20:55:08.931925 (+ 3596us) write_op.cc:270] Start()
1211 20:55:08.931947 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497025482 usec, L: 0
1211 20:55:08.933804 (+ 1857us) write_op.cc:270] Start()
1211 20:55:08.933829 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497040914 usec, L: 0
1211 20:55:08.944442 (+ 10613us) write_op.cc:270] Start()
1211 20:55:08.944467 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497055535 usec, L: 0
1211 20:55:08.944630 (+ 163us) write_op.cc:270] Start()
1211 20:55:08.944692 (+ 62us) write_op.cc:276] Timestamp: P: 1765486497074690 usec, L: 0
1211 20:55:08.954150 (+ 9458us) write_op.cc:270] Start()
1211 20:55:08.954170 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497083184 usec, L: 0
1211 20:55:08.954297 (+ 127us) write_op.cc:270] Start()
1211 20:55:08.954315 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497085591 usec, L: 0
1211 20:55:08.954489 (+ 174us) write_op.cc:270] Start()
1211 20:55:08.954512 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497086233 usec, L: 0
1211 20:55:08.954659 (+ 147us) write_op.cc:270] Start()
1211 20:55:08.954681 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497114905 usec, L: 0
1211 20:55:08.954819 (+ 138us) write_op.cc:270] Start()
1211 20:55:08.954838 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497134308 usec, L: 0
1211 20:55:08.954967 (+ 129us) write_op.cc:270] Start()
1211 20:55:08.954985 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497137007 usec, L: 0
1211 20:55:08.955117 (+ 132us) write_op.cc:270] Start()
1211 20:55:08.955138 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497143938 usec, L: 0
1211 20:55:08.955268 (+ 130us) write_op.cc:270] Start()
1211 20:55:08.955286 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497165767 usec, L: 0
1211 20:55:08.955536 (+ 250us) write_op.cc:270] Start()
1211 20:55:08.955557 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497178899 usec, L: 0
1211 20:55:08.955710 (+ 153us) write_op.cc:270] Start()
1211 20:55:08.955727 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497194828 usec, L: 0
1211 20:55:08.955887 (+ 160us) write_op.cc:270] Start()
1211 20:55:08.955915 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497210509 usec, L: 0
1211 20:55:08.956077 (+ 162us) write_op.cc:270] Start()
1211 20:55:08.956099 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497215073 usec, L: 0
1211 20:55:08.956243 (+ 144us) write_op.cc:270] Start()
1211 20:55:08.956262 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497228464 usec, L: 0
1211 20:55:08.956432 (+ 170us) write_op.cc:270] Start()
1211 20:55:08.956456 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497239486 usec, L: 0
1211 20:55:08.956633 (+ 177us) write_op.cc:270] Start()
1211 20:55:08.956656 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497264450 usec, L: 0
1211 20:55:08.956797 (+ 141us) write_op.cc:270] Start()
1211 20:55:08.956821 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497268642 usec, L: 0
1211 20:55:08.956973 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.956995 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497291621 usec, L: 0
1211 20:55:08.957127 (+ 132us) write_op.cc:270] Start()
1211 20:55:08.957146 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497303448 usec, L: 0
1211 20:55:08.964985 (+ 7839us) write_op.cc:270] Start()
1211 20:55:08.965002 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497324133 usec, L: 0
1211 20:55:08.965130 (+ 128us) write_op.cc:270] Start()
1211 20:55:08.965145 (+ 15us) write_op.cc:276] Timestamp: P: 1765486497349281 usec, L: 0
1211 20:55:08.965295 (+ 150us) write_op.cc:270] Start()
1211 20:55:08.965316 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497357023 usec, L: 0
1211 20:55:08.965480 (+ 164us) write_op.cc:270] Start()
1211 20:55:08.965501 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497364021 usec, L: 0
1211 20:55:08.965646 (+ 145us) write_op.cc:270] Start()
1211 20:55:08.965666 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497379652 usec, L: 0
1211 20:55:08.965820 (+ 154us) write_op.cc:270] Start()
1211 20:55:08.965840 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497400253 usec, L: 0
1211 20:55:08.966028 (+ 188us) write_op.cc:270] Start()
1211 20:55:08.966050 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497402785 usec, L: 0
1211 20:55:08.966183 (+ 133us) write_op.cc:270] Start()
1211 20:55:08.966202 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497415033 usec, L: 0
1211 20:55:08.966367 (+ 165us) write_op.cc:270] Start()
1211 20:55:08.966401 (+ 34us) write_op.cc:276] Timestamp: P: 1765486497439172 usec, L: 0
1211 20:55:08.966567 (+ 166us) write_op.cc:270] Start()
1211 20:55:08.966591 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497443970 usec, L: 0
1211 20:55:08.966723 (+ 132us) write_op.cc:270] Start()
1211 20:55:08.966742 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497447324 usec, L: 0
1211 20:55:08.966885 (+ 143us) write_op.cc:270] Start()
1211 20:55:08.966904 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497458514 usec, L: 0
1211 20:55:08.967092 (+ 188us) write_op.cc:270] Start()
1211 20:55:08.967115 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497493950 usec, L: 0
1211 20:55:08.967264 (+ 149us) write_op.cc:270] Start()
1211 20:55:08.967284 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497495343 usec, L: 0
1211 20:55:08.967452 (+ 168us) write_op.cc:270] Start()
1211 20:55:08.967474 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497508763 usec, L: 0
1211 20:55:08.967631 (+ 157us) write_op.cc:270] Start()
1211 20:55:08.967651 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497515676 usec, L: 0
1211 20:55:08.967822 (+ 171us) write_op.cc:270] Start()
1211 20:55:08.967841 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497539027 usec, L: 0
1211 20:55:08.968030 (+ 189us) write_op.cc:270] Start()
1211 20:55:08.968050 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497544373 usec, L: 0
1211 20:55:08.968202 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.968222 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497554845 usec, L: 0
1211 20:55:08.968390 (+ 168us) write_op.cc:270] Start()
1211 20:55:08.968412 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497557560 usec, L: 0
1211 20:55:08.968562 (+ 150us) write_op.cc:270] Start()
1211 20:55:08.968582 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497584974 usec, L: 0
1211 20:55:08.968743 (+ 161us) write_op.cc:270] Start()
1211 20:55:08.968764 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497597578 usec, L: 0
1211 20:55:08.968918 (+ 154us) write_op.cc:270] Start()
1211 20:55:08.968938 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497608377 usec, L: 0
1211 20:55:08.969082 (+ 144us) write_op.cc:270] Start()
1211 20:55:08.969102 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497641799 usec, L: 0
1211 20:55:08.969241 (+ 139us) write_op.cc:270] Start()
1211 20:55:08.969263 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497643702 usec, L: 0
1211 20:55:08.969415 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.969434 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497655617 usec, L: 0
1211 20:55:08.969565 (+ 131us) write_op.cc:270] Start()
1211 20:55:08.969583 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497666524 usec, L: 0
1211 20:55:08.969712 (+ 129us) write_op.cc:270] Start()
1211 20:55:08.969728 (+ 16us) write_op.cc:276] Timestamp: P: 1765486497684058 usec, L: 0
1211 20:55:08.978099 (+ 8371us) write_op.cc:270] Start()
1211 20:55:08.978122 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497685344 usec, L: 0
1211 20:55:08.978300 (+ 178us) write_op.cc:270] Start()
1211 20:55:08.978347 (+ 47us) write_op.cc:276] Timestamp: P: 1765486497710058 usec, L: 0
1211 20:55:08.978530 (+ 183us) write_op.cc:270] Start()
1211 20:55:08.978554 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497720564 usec, L: 0
1211 20:55:08.978736 (+ 182us) write_op.cc:270] Start()
1211 20:55:08.978759 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497735188 usec, L: 0
1211 20:55:08.978911 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.978933 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497745157 usec, L: 0
1211 20:55:08.979109 (+ 176us) write_op.cc:270] Start()
1211 20:55:08.979129 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497761893 usec, L: 0
1211 20:55:08.986101 (+ 6972us) write_op.cc:270] Start()
1211 20:55:08.986128 (+ 27us) write_op.cc:276] Timestamp: P: 1765486497775232 usec, L: 0
1211 20:55:08.986301 (+ 173us) write_op.cc:270] Start()
1211 20:55:08.986323 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497782097 usec, L: 0
1211 20:55:08.986478 (+ 155us) write_op.cc:270] Start()
1211 20:55:08.986498 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497803609 usec, L: 0
1211 20:55:08.986665 (+ 167us) write_op.cc:270] Start()
1211 20:55:08.986687 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497819703 usec, L: 0
1211 20:55:08.986836 (+ 149us) write_op.cc:270] Start()
1211 20:55:08.986856 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497821755 usec, L: 0
1211 20:55:08.987043 (+ 187us) write_op.cc:270] Start()
1211 20:55:08.987065 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497843684 usec, L: 0
1211 20:55:08.987217 (+ 152us) write_op.cc:270] Start()
1211 20:55:08.987239 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497855355 usec, L: 0
1211 20:55:08.987465 (+ 226us) write_op.cc:270] Start()
1211 20:55:08.987487 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497869818 usec, L: 0
1211 20:55:08.987627 (+ 140us) write_op.cc:270] Start()
1211 20:55:08.987645 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497884581 usec, L: 0
1211 20:55:08.987770 (+ 125us) write_op.cc:270] Start()
1211 20:55:08.987787 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497886717 usec, L: 0
1211 20:55:08.987957 (+ 170us) write_op.cc:270] Start()
1211 20:55:08.987976 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497911909 usec, L: 0
1211 20:55:08.988116 (+ 140us) write_op.cc:270] Start()
1211 20:55:08.988141 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497929266 usec, L: 0
1211 20:55:08.988317 (+ 176us) write_op.cc:270] Start()
1211 20:55:08.988333 (+ 16us) write_op.cc:276] Timestamp: P: 1765486497943179 usec, L: 0
1211 20:55:08.990109 (+ 1776us) write_op.cc:270] Start()
1211 20:55:08.990146 (+ 37us) write_op.cc:276] Timestamp: P: 1765486497951514 usec, L: 0
1211 20:55:08.990282 (+ 136us) write_op.cc:270] Start()
1211 20:55:08.990988 (+ 706us) write_op.cc:276] Timestamp: P: 1765486497961880 usec, L: 0
1211 20:55:08.991253 (+ 265us) write_op.cc:270] Start()
1211 20:55:08.991282 (+ 29us) write_op.cc:276] Timestamp: P: 1765486497986103 usec, L: 0
1211 20:55:08.991446 (+ 164us) write_op.cc:270] Start()
1211 20:55:08.991474 (+ 28us) write_op.cc:276] Timestamp: P: 1765486498010402 usec, L: 0
1211 20:55:08.991641 (+ 167us) write_op.cc:270] Start()
1211 20:55:08.991667 (+ 26us) write_op.cc:276] Timestamp: P: 1765486498029423 usec, L: 0
1211 20:55:08.991850 (+ 183us) write_op.cc:270] Start()
1211 20:55:08.991905 (+ 55us) write_op.cc:276] Timestamp: P: 1765486498034784 usec, L: 0
1211 20:55:08.992084 (+ 179us) write_op.cc:270] Start()
1211 20:55:08.992107 (+ 23us) write_op.cc:276] Timestamp: P: 1765486498040162 usec, L: 0
1211 20:55:08.992338 (+ 231us) write_op.cc:270] Start()
1211 20:55:08.992361 (+ 23us) write_op.cc:276] Timestamp: P: 1765486498047333 usec, L: 0
1211 20:55:08.992527 (+ 166us) write_op.cc:270] Start()
1211 20:55:08.992610 (+ 83us) write_op.cc:276] Timestamp: P: 1765486498076479 usec, L: 0
1211 20:55:08.992861 (+ 251us) write_op.cc:270] Start()
1211 20:55:08.992885 (+ 24us) write_op.cc:276] Timestamp: P: 1765486498093355 usec, L: 0
1211 20:55:08.993079 (+ 194us) write_op.cc:270] Start()
1211 20:55:08.993103 (+ 24us) write_op.cc:276] Timestamp: P: 1765486498110505 usec, L: 0
1211 20:55:08.993269 (+ 166us) write_op.cc:270] Start()
1211 20:55:08.993292 (+ 23us) write_op.cc:276] Timestamp: P: 1765486498117520 usec, L: 0
1211 20:55:08.993462 (+ 170us) write_op.cc:270] Start()
1211 20:55:08.993485 (+ 23us) write_op.cc:276] Timestamp: P: 1765486498131982 usec, L: 0
1211 20:55:08.993645 (+ 160us) write_op.cc:270] Start()
1211 20:55:08.993667 (+ 22us) write_op.cc:276] Timestamp: P: 1765486498135324 usec, L: 0
1211 20:55:08.993829 (+ 162us) write_op.cc:270] Start()
1211 20:55:08.993851 (+ 22us) write_op.cc:276] Timestamp: P: 1765486498162332 usec, L: 0
1211 20:55:09.021833 (+ 27982us) write_op.cc:270] Start()
1211 20:55:09.021874 (+ 41us) write_op.cc:276] Timestamp: P: 1765486498181288 usec, L: 0
1211 20:55:09.022051 (+ 177us) write_op.cc:270] Start()
1211 20:55:09.022073 (+ 22us) write_op.cc:276] Timestamp: P: 1765486498184746 usec, L: 0
1211 20:55:09.022285 (+ 212us) write_op.cc:270] Start()
1211 20:55:09.022306 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498191459 usec, L: 0
1211 20:55:09.022463 (+ 157us) write_op.cc:270] Start()
1211 20:55:09.022483 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498214108 usec, L: 0
1211 20:55:09.022628 (+ 145us) write_op.cc:270] Start()
1211 20:55:09.022647 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498232237 usec, L: 0
1211 20:55:09.022824 (+ 177us) write_op.cc:270] Start()
1211 20:55:09.022845 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498241915 usec, L: 0
1211 20:55:09.022999 (+ 154us) write_op.cc:270] Start()
1211 20:55:09.023020 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498259817 usec, L: 0
1211 20:55:09.023173 (+ 153us) write_op.cc:270] Start()
1211 20:55:09.023195 (+ 22us) write_op.cc:276] Timestamp: P: 1765486498274975 usec, L: 0
1211 20:55:09.023343 (+ 148us) write_op.cc:270] Start()
1211 20:55:09.023363 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498283010 usec, L: 0
1211 20:55:09.023519 (+ 156us) write_op.cc:270] Start()
1211 20:55:09.023536 (+ 17us) write_op.cc:276] Timestamp: P: 1765486498304832 usec, L: 0
1211 20:55:09.023680 (+ 144us) write_op.cc:270] Start()
1211 20:55:09.023718 (+ 38us) write_op.cc:276] Timestamp: P: 1765486498309906 usec, L: 0
1211 20:55:09.023863 (+ 145us) write_op.cc:270] Start()
1211 20:55:09.023885 (+ 22us) write_op.cc:276] Timestamp: P: 1765486498322926 usec, L: 0
1211 20:55:09.024023 (+ 138us) write_op.cc:270] Start()
1211 20:55:09.024041 (+ 18us) write_op.cc:276] Timestamp: P: 1765486498333558 usec, L: 0
1211 20:55:09.024178 (+ 137us) write_op.cc:270] Start()
1211 20:55:09.024196 (+ 18us) write_op.cc:276] Timestamp: P: 1765486498343661 usec, L: 0
1211 20:55:09.024324 (+ 128us) write_op.cc:270] Start()
1211 20:55:09.024341 (+ 17us) write_op.cc:276] Timestamp: P: 1765486498382429 usec, L: 0
1211 20:55:09.024475 (+ 134us) write_op.cc:270] Start()
1211 20:55:09.024512 (+ 37us) write_op.cc:276] Timestamp: P: 1765486498383824 usec, L: 0
1211 20:55:09.024665 (+ 153us) write_op.cc:270] Start()
1211 20:55:09.024683 (+ 18us) write_op.cc:276] Timestamp: P: 1765486498388128 usec, L: 0
1211 20:55:09.024868 (+ 185us) write_op.cc:270] Start()
1211 20:55:09.024886 (+ 18us) write_op.cc:276] Timestamp: P: 1765486498414424 usec, L: 0
1211 20:55:09.025043 (+ 157us) write_op.cc:270] Start()
1211 20:55:09.025062 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498429833 usec, L: 0
1211 20:55:09.025189 (+ 127us) write_op.cc:270] Start()
1211 20:55:09.025206 (+ 17us) write_op.cc:276] Timestamp: P: 1765486498431801 usec, L: 0
1211 20:55:09.025319 (+ 113us) write_op.cc:270] Start()
1211 20:55:09.025335 (+ 16us) write_op.cc:276] Timestamp: P: 1765486498447604 usec, L: 0
1211 20:55:09.025466 (+ 131us) write_op.cc:270] Start()
1211 20:55:09.025484 (+ 18us) write_op.cc
I20251211 20:55:15.111145 7057 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:15.111335 7057 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:15.112658 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 986 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:15.125849 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 986 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:15.140271 7057 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:15.140487 6872 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:15.140964 7057 raft_consensus.cc:2749] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:15.249358 7057 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:15.249522 7057 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:15.254128 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1232 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:15.257454 7057 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:15.283078 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1232 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:15.334586 6872 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:15.482668 7057 raft_consensus.cc:2749] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:15.706086 7057 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:15.706245 7057 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:15.708144 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 986 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:15.709347 7057 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:15.712265 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 986 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:15.713268 6872 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:15.713716 7057 raft_consensus.cc:2749] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:15.736670 7015 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:15.748530 7015 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:15.749333 7015 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:15.752538 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1180 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:15.753446 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1180 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:15.754772 6872 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:15.755491 7015 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:15.778651 6941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.194:47609 (request call id 14) took 1859 ms. Trace:
I20251211 20:55:15.896116 6875 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Bootstrap replayed 1/1 log segments. Stats: ops{read=160 overwritten=0 applied=159 ignored=0} inserts{seen=1289 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251211 20:55:15.920830 6875 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Bootstrap complete.
I20251211 20:55:15.946583 6875 ts_tablet_manager.cc:1403] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Time spent bootstrapping tablet: real 5.732s user 0.287s sys 0.075s
I20251211 20:55:15.957728 6875 raft_consensus.cc:359] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:16.000452 6875 raft_consensus.cc:740] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd482952c9fd4a1fb75c31f128338e89, State: Initialized, Role: FOLLOWER
I20251211 20:55:16.021040 6875 consensus_queue.cc:260] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 159, Last appended: 1.160, Last appended by leader: 160, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:16.021610 6875 ts_tablet_manager.cc:1434] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89: Time spent starting tablet: real 0.075s user 0.003s sys 0.000s
I20251211 20:55:16.145570 7015 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:16.149736 7015 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:16.151496 7015 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:16.164181 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1180 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:16.165258 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1180 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:16.165853 6872 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:16.171887 7015 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:15.791762 6941 rpcz_store.cc:276] 1211 20:55:13.871137 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:13.871828 (+ 691us) service_pool.cc:225] Handling call
1211 20:55:14.024659 (+152831us) raft_consensus.cc:1471] Updating replica for 302 ops
1211 20:55:14.109108 (+ 84449us) raft_consensus.cc:1534] Early marking committed up to index 878
1211 20:55:14.109214 (+ 106us) raft_consensus.cc:1539] Triggering prepare for 302 ops
1211 20:55:14.109372 (+ 158us) write_op.cc:270] Start()
1211 20:55:14.109397 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508124428 usec, L: 0
1211 20:55:14.109586 (+ 189us) write_op.cc:270] Start()
1211 20:55:14.109624 (+ 38us) write_op.cc:276] Timestamp: P: 1765486508135118 usec, L: 0
1211 20:55:14.109830 (+ 206us) write_op.cc:270] Start()
1211 20:55:14.109849 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508144580 usec, L: 0
1211 20:55:14.110011 (+ 162us) write_op.cc:270] Start()
1211 20:55:14.110039 (+ 28us) write_op.cc:276] Timestamp: P: 1765486508166245 usec, L: 0
1211 20:55:14.110237 (+ 198us) write_op.cc:270] Start()
1211 20:55:14.110257 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508192423 usec, L: 0
1211 20:55:14.110458 (+ 201us) write_op.cc:270] Start()
1211 20:55:14.110479 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508199159 usec, L: 0
1211 20:55:14.110712 (+ 233us) write_op.cc:270] Start()
1211 20:55:14.110779 (+ 67us) write_op.cc:276] Timestamp: P: 1765486508208016 usec, L: 0
1211 20:55:14.133059 (+ 22280us) write_op.cc:270] Start()
1211 20:55:14.133083 (+ 24us) write_op.cc:276] Timestamp: P: 1765486508221804 usec, L: 0
1211 20:55:14.133908 (+ 825us) write_op.cc:270] Start()
1211 20:55:14.133929 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508262544 usec, L: 0
1211 20:55:14.134864 (+ 935us) write_op.cc:270] Start()
1211 20:55:14.134884 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508264592 usec, L: 0
1211 20:55:14.181554 (+ 46670us) write_op.cc:270] Start()
1211 20:55:14.181574 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508275190 usec, L: 0
1211 20:55:14.181772 (+ 198us) write_op.cc:270] Start()
1211 20:55:14.181790 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508279382 usec, L: 0
1211 20:55:14.182043 (+ 253us) write_op.cc:270] Start()
1211 20:55:14.182064 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508318375 usec, L: 0
1211 20:55:14.200132 (+ 18068us) write_op.cc:270] Start()
1211 20:55:14.200155 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508328394 usec, L: 0
1211 20:55:14.200330 (+ 175us) write_op.cc:270] Start()
1211 20:55:14.200349 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508332782 usec, L: 0
1211 20:55:14.200524 (+ 175us) write_op.cc:270] Start()
1211 20:55:14.200542 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508346336 usec, L: 0
1211 20:55:14.200683 (+ 141us) write_op.cc:270] Start()
1211 20:55:14.200700 (+ 17us) write_op.cc:276] Timestamp: P: 1765486508353246 usec, L: 0
1211 20:55:14.204179 (+ 3479us) write_op.cc:270] Start()
1211 20:55:14.204199 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508393938 usec, L: 0
1211 20:55:14.204394 (+ 195us) write_op.cc:270] Start()
1211 20:55:14.204411 (+ 17us) write_op.cc:276] Timestamp: P: 1765486508394455 usec, L: 0
1211 20:55:14.204550 (+ 139us) write_op.cc:270] Start()
1211 20:55:14.204565 (+ 15us) write_op.cc:276] Timestamp: P: 1765486508395799 usec, L: 0
1211 20:55:14.204686 (+ 121us) write_op.cc:270] Start()
1211 20:55:14.204701 (+ 15us) write_op.cc:276] Timestamp: P: 1765486508402082 usec, L: 0
1211 20:55:14.204830 (+ 129us) write_op.cc:270] Start()
1211 20:55:14.204844 (+ 14us) write_op.cc:276] Timestamp: P: 1765486508445948 usec, L: 0
1211 20:55:14.205035 (+ 191us) write_op.cc:270] Start()
1211 20:55:14.205052 (+ 17us) write_op.cc:276] Timestamp: P: 1765486508473157 usec, L: 0
1211 20:55:14.205209 (+ 157us) write_op.cc:270] Start()
1211 20:55:14.205226 (+ 17us) write_op.cc:276] Timestamp: P: 1765486508476606 usec, L: 0
1211 20:55:14.205392 (+ 166us) write_op.cc:270] Start()
1211 20:55:14.205411 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508496830 usec, L: 0
1211 20:55:14.205574 (+ 163us) write_op.cc:270] Start()
1211 20:55:14.205592 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508510829 usec, L: 0
1211 20:55:14.205742 (+ 150us) write_op.cc:270] Start()
1211 20:55:14.205759 (+ 17us) write_op.cc:276] Timestamp: P: 1765486508512220 usec, L: 0
1211 20:55:14.205912 (+ 153us) write_op.cc:270] Start()
1211 20:55:14.205950 (+ 38us) write_op.cc:276] Timestamp: P: 1765486508527701 usec, L: 0
1211 20:55:14.207053 (+ 1103us) write_op.cc:270] Start()
1211 20:55:14.207077 (+ 24us) write_op.cc:276] Timestamp: P: 1765486508556231 usec, L: 0
1211 20:55:14.207261 (+ 184us) write_op.cc:270] Start()
1211 20:55:14.207280 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508564707 usec, L: 0
1211 20:55:14.207461 (+ 181us) write_op.cc:270] Start()
1211 20:55:14.207486 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508579406 usec, L: 0
1211 20:55:14.207663 (+ 177us) write_op.cc:270] Start()
1211 20:55:14.207683 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508607062 usec, L: 0
1211 20:55:14.208656 (+ 973us) write_op.cc:270] Start()
1211 20:55:14.208680 (+ 24us) write_op.cc:276] Timestamp: P: 1765486508629549 usec, L: 0
1211 20:55:14.208856 (+ 176us) write_op.cc:270] Start()
1211 20:55:14.208891 (+ 35us) write_op.cc:276] Timestamp: P: 1765486508636277 usec, L: 0
1211 20:55:14.209056 (+ 165us) write_op.cc:270] Start()
1211 20:55:14.209074 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508653695 usec, L: 0
1211 20:55:14.209779 (+ 705us) write_op.cc:270] Start()
1211 20:55:14.209800 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508666654 usec, L: 0
1211 20:55:14.210719 (+ 919us) write_op.cc:270] Start()
1211 20:55:14.210739 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508702790 usec, L: 0
1211 20:55:14.217814 (+ 7075us) write_op.cc:270] Start()
1211 20:55:14.217837 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508723534 usec, L: 0
1211 20:55:14.220358 (+ 2521us) write_op.cc:270] Start()
1211 20:55:14.220380 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508724152 usec, L: 0
1211 20:55:14.221154 (+ 774us) write_op.cc:270] Start()
1211 20:55:14.221202 (+ 48us) write_op.cc:276] Timestamp: P: 1765486508783712 usec, L: 0
1211 20:55:14.221395 (+ 193us) write_op.cc:270] Start()
1211 20:55:14.221420 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508820014 usec, L: 0
1211 20:55:14.228522 (+ 7102us) write_op.cc:270] Start()
1211 20:55:14.228546 (+ 24us) write_op.cc:276] Timestamp: P: 1765486508822359 usec, L: 0
1211 20:55:14.230128 (+ 1582us) write_op.cc:270] Start()
1211 20:55:14.230151 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508852973 usec, L: 0
1211 20:55:14.230915 (+ 764us) write_op.cc:270] Start()
1211 20:55:14.230958 (+ 43us) write_op.cc:276] Timestamp: P: 1765486508877330 usec, L: 0
1211 20:55:14.231192 (+ 234us) write_op.cc:270] Start()
1211 20:55:14.231213 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508902027 usec, L: 0
1211 20:55:14.231388 (+ 175us) write_op.cc:270] Start()
1211 20:55:14.231414 (+ 26us) write_op.cc:276] Timestamp: P: 1765486508906871 usec, L: 0
1211 20:55:14.231592 (+ 178us) write_op.cc:270] Start()
1211 20:55:14.231610 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508918227 usec, L: 0
1211 20:55:14.232396 (+ 786us) write_op.cc:270] Start()
1211 20:55:14.232419 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508967896 usec, L: 0
1211 20:55:14.232625 (+ 206us) write_op.cc:270] Start()
1211 20:55:14.232646 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509003745 usec, L: 0
1211 20:55:14.232876 (+ 230us) write_op.cc:270] Start()
1211 20:55:14.232897 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509007820 usec, L: 0
1211 20:55:14.234844 (+ 1947us) write_op.cc:270] Start()
1211 20:55:14.234867 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509054387 usec, L: 0
1211 20:55:14.235068 (+ 201us) write_op.cc:270] Start()
1211 20:55:14.235089 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509054908 usec, L: 0
1211 20:55:14.235292 (+ 203us) write_op.cc:270] Start()
1211 20:55:14.235314 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509082963 usec, L: 0
1211 20:55:14.235519 (+ 205us) write_op.cc:270] Start()
1211 20:55:14.235540 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509123713 usec, L: 0
1211 20:55:14.235810 (+ 270us) write_op.cc:270] Start()
1211 20:55:14.235835 (+ 25us) write_op.cc:276] Timestamp: P: 1765486509143565 usec, L: 0
1211 20:55:14.236043 (+ 208us) write_op.cc:270] Start()
1211 20:55:14.236070 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509161401 usec, L: 0
1211 20:55:14.236264 (+ 194us) write_op.cc:270] Start()
1211 20:55:14.236287 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509172962 usec, L: 0
1211 20:55:14.236497 (+ 210us) write_op.cc:270] Start()
1211 20:55:14.236518 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509199126 usec, L: 0
1211 20:55:14.236718 (+ 200us) write_op.cc:270] Start()
1211 20:55:14.236741 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509216680 usec, L: 0
1211 20:55:14.236966 (+ 225us) write_op.cc:270] Start()
1211 20:55:14.237026 (+ 60us) write_op.cc:276] Timestamp: P: 1765486509219426 usec, L: 0
1211 20:55:14.237262 (+ 236us) write_op.cc:270] Start()
1211 20:55:14.237285 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509253633 usec, L: 0
1211 20:55:14.251035 (+ 13750us) write_op.cc:270] Start()
1211 20:55:14.251063 (+ 28us) write_op.cc:276] Timestamp: P: 1765486509262553 usec, L: 0
1211 20:55:14.251271 (+ 208us) write_op.cc:270] Start()
1211 20:55:14.251293 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509285097 usec, L: 0
1211 20:55:14.251492 (+ 199us) write_op.cc:270] Start()
1211 20:55:14.251512 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509300609 usec, L: 0
1211 20:55:14.259801 (+ 8289us) write_op.cc:270] Start()
1211 20:55:14.259835 (+ 34us) write_op.cc:276] Timestamp: P: 1765486509324468 usec, L: 0
1211 20:55:14.260051 (+ 216us) write_op.cc:270] Start()
1211 20:55:14.260075 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509344830 usec, L: 0
1211 20:55:14.260266 (+ 191us) write_op.cc:270] Start()
1211 20:55:14.260292 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509351943 usec, L: 0
1211 20:55:14.260473 (+ 181us) write_op.cc:270] Start()
1211 20:55:14.260495 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509369021 usec, L: 0
1211 20:55:14.260681 (+ 186us) write_op.cc:270] Start()
1211 20:55:14.260705 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509384686 usec, L: 0
1211 20:55:14.260875 (+ 170us) write_op.cc:270] Start()
1211 20:55:14.260898 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509410153 usec, L: 0
1211 20:55:14.261100 (+ 202us) write_op.cc:270] Start()
1211 20:55:14.261121 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509429542 usec, L: 0
1211 20:55:14.261328 (+ 207us) write_op.cc:270] Start()
1211 20:55:14.261352 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509434623 usec, L: 0
1211 20:55:14.272914 (+ 11562us) write_op.cc:270] Start()
1211 20:55:14.272940 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509445842 usec, L: 0
1211 20:55:14.275066 (+ 2126us) write_op.cc:270] Start()
1211 20:55:14.275092 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509450563 usec, L: 0
1211 20:55:14.278549 (+ 3457us) write_op.cc:270] Start()
1211 20:55:14.278571 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509469063 usec, L: 0
1211 20:55:14.278727 (+ 156us) write_op.cc:270] Start()
1211 20:55:14.287026 (+ 8299us) write_op.cc:276] Timestamp: P: 1765486509482963 usec, L: 0
1211 20:55:14.306292 (+ 19266us) write_op.cc:270] Start()
1211 20:55:14.306319 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509501403 usec, L: 0
1211 20:55:14.306546 (+ 227us) write_op.cc:270] Start()
1211 20:55:14.306568 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509518955 usec, L: 0
1211 20:55:14.306718 (+ 150us) write_op.cc:270] Start()
1211 20:55:14.306737 (+ 19us) write_op.cc:276] Timestamp: P: 1765486509523198 usec, L: 0
1211 20:55:14.306880 (+ 143us) write_op.cc:270] Start()
1211 20:55:14.306898 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509545654 usec, L: 0
1211 20:55:14.307102 (+ 204us) write_op.cc:270] Start()
1211 20:55:14.307126 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509567939 usec, L: 0
1211 20:55:14.307300 (+ 174us) write_op.cc:270] Start()
1211 20:55:14.307321 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509580696 usec, L: 0
1211 20:55:14.307471 (+ 150us) write_op.cc:270] Start()
1211 20:55:14.307491 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509583191 usec, L: 0
1211 20:55:14.307624 (+ 133us) write_op.cc:270] Start()
1211 20:55:14.307644 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509597795 usec, L: 0
1211 20:55:14.307797 (+ 153us) write_op.cc:270] Start()
1211 20:55:14.307818 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509620754 usec, L: 0
1211 20:55:14.307952 (+ 134us) write_op.cc:270] Start()
1211 20:55:14.307972 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509658860 usec, L: 0
1211 20:55:14.308135 (+ 163us) write_op.cc:270] Start()
1211 20:55:14.308168 (+ 33us) write_op.cc:276] Timestamp: P: 1765486509678883 usec, L: 0
1211 20:55:14.308333 (+ 165us) write_op.cc:270] Start()
1211 20:55:14.308359 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509684226 usec, L: 0
1211 20:55:14.308485 (+ 126us) write_op.cc:270] Start()
1211 20:55:14.308502 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509708129 usec, L: 0
1211 20:55:14.308641 (+ 139us) write_op.cc:270] Start()
1211 20:55:14.308661 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509732386 usec, L: 0
1211 20:55:14.322660 (+ 13999us) write_op.cc:270] Start()
1211 20:55:14.322705 (+ 45us) write_op.cc:276] Timestamp: P: 1765486509732919 usec, L: 0
1211 20:55:14.326811 (+ 4106us) write_op.cc:270] Start()
1211 20:55:14.338528 (+ 11717us) write_op.cc:276] Timestamp: P: 1765486509786577 usec, L: 0
1211 20:55:14.347887 (+ 9359us) write_op.cc:270] Start()
1211 20:55:14.347914 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509801842 usec, L: 0
1211 20:55:14.354896 (+ 6982us) write_op.cc:270] Start()
1211 20:55:14.354922 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509816288 usec, L: 0
1211 20:55:14.355298 (+ 376us) write_op.cc:270] Start()
1211 20:55:14.355318 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509817193 usec, L: 0
1211 20:55:14.405909 (+ 50591us) write_op.cc:270] Start()
1211 20:55:14.405946 (+ 37us) write_op.cc:276] Timestamp: P: 1765486509851078 usec, L: 0
1211 20:55:14.407679 (+ 1733us) write_op.cc:270] Start()
1211 20:55:14.407706 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509851658 usec, L: 0
1211 20:55:14.407914 (+ 208us) write_op.cc:270] Start()
1211 20:55:14.407936 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509894602 usec, L: 0
1211 20:55:14.415739 (+ 7803us) write_op.cc:270] Start()
1211 20:55:14.415762 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509910059 usec, L: 0
1211 20:55:14.419014 (+ 3252us) write_op.cc:270] Start()
1211 20:55:14.419038 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509916407 usec, L: 0
1211 20:55:14.431894 (+ 12856us) write_op.cc:270] Start()
1211 20:55:14.431923 (+ 29us) write_op.cc:276] Timestamp: P: 1765486509928669 usec, L: 0
1211 20:55:14.464214 (+ 32291us) write_op.cc:270] Start()
1211 20:55:14.464245 (+ 31us) write_op.cc:276] Timestamp: P: 1765486509962058 usec, L: 0
1211 20:55:14.483940 (+ 19695us) write_op.cc:270] Start()
1211 20:55:14.483966 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509983440 usec, L: 0
1211 20:55:14.484898 (+ 932us) write_op.cc:270] Start()
1211 20:55:14.484920 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509984102 usec, L: 0
1211 20:55:14.488883 (+ 3963us) write_op.cc:270] Start()
1211 20:55:14.488909 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509984708 usec, L: 0
1211 20:55:14.491821 (+ 2912us) write_op.cc:270] Start()
1211 20:55:14.491847 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510028929 usec, L: 0
1211 20:55:14.494646 (+ 2799us) write_op.cc:270] Start()
1211 20:55:14.494668 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510038476 usec, L: 0
1211 20:55:14.502469 (+ 7801us) write_op.cc:270] Start()
1211 20:55:14.524445 (+ 21976us) write_op.cc:276] Timestamp: P: 1765486510070842 usec, L: 0
1211 20:55:14.538766 (+ 14321us) write_op.cc:270] Start()
1211 20:55:14.538789 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510092865 usec, L: 0
1211 20:55:14.551369 (+ 12580us) write_op.cc:270] Start()
1211 20:55:14.551398 (+ 29us) write_op.cc:276] Timestamp: P: 1765486510108965 usec, L: 0
1211 20:55:14.560902 (+ 9504us) write_op.cc:270] Start()
1211 20:55:14.560928 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510109705 usec, L: 0
1211 20:55:14.565408 (+ 4480us) write_op.cc:270] Start()
1211 20:55:14.565450 (+ 42us) write_op.cc:276] Timestamp: P: 1765486510160615 usec, L: 0
1211 20:55:14.597505 (+ 32055us) write_op.cc:270] Start()
1211 20:55:14.597533 (+ 28us) write_op.cc:276] Timestamp: P: 1765486510211286 usec, L: 0
1211 20:55:14.599205 (+ 1672us) write_op.cc:270] Start()
1211 20:55:14.599229 (+ 24us) write_op.cc:276] Timestamp: P: 1765486510212061 usec, L: 0
1211 20:55:14.606495 (+ 7266us) write_op.cc:270] Start()
1211 20:55:14.606523 (+ 28us) write_op.cc:276] Timestamp: P: 1765486510230102 usec, L: 0
1211 20:55:14.606694 (+ 171us) write_op.cc:270] Start()
1211 20:55:14.606715 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510235200 usec, L: 0
1211 20:55:14.613902 (+ 7187us) write_op.cc:270] Start()
1211 20:55:14.613923 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510288640 usec, L: 0
1211 20:55:14.621497 (+ 7574us) write_op.cc:270] Start()
1211 20:55:14.621525 (+ 28us) write_op.cc:276] Timestamp: P: 1765486510313798 usec, L: 0
1211 20:55:14.621733 (+ 208us) write_op.cc:270] Start()
1211 20:55:14.621754 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510332486 usec, L: 0
1211 20:55:14.621945 (+ 191us) write_op.cc:270] Start()
1211 20:55:14.621963 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510360904 usec, L: 0
1211 20:55:14.622157 (+ 194us) write_op.cc:270] Start()
1211 20:55:14.622177 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510389154 usec, L: 0
1211 20:55:14.622319 (+ 142us) write_op.cc:270] Start()
1211 20:55:14.622345 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510398235 usec, L: 0
1211 20:55:14.622519 (+ 174us) write_op.cc:270] Start()
1211 20:55:14.622538 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510424136 usec, L: 0
1211 20:55:14.651878 (+ 29340us) write_op.cc:270] Start()
1211 20:55:14.660121 (+ 8243us) write_op.cc:276] Timestamp: P: 1765486510439340 usec, L: 0
1211 20:55:14.660381 (+ 260us) write_op.cc:270] Start()
1211 20:55:14.660403 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510480864 usec, L: 0
1211 20:55:14.660551 (+ 148us) write_op.cc:270] Start()
1211 20:55:14.660571 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510483601 usec, L: 0
1211 20:55:14.660703 (+ 132us) write_op.cc:270] Start()
1211 20:55:14.660721 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510489985 usec, L: 0
1211 20:55:14.676066 (+ 15345us) write_op.cc:270] Start()
1211 20:55:14.676092 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510519754 usec, L: 0
1211 20:55:14.677845 (+ 1753us) write_op.cc:270] Start()
1211 20:55:14.677880 (+ 35us) write_op.cc:276] Timestamp: P: 1765486510556759 usec, L: 0
1211 20:55:14.678123 (+ 243us) write_op.cc:270] Start()
1211 20:55:14.678158 (+ 35us) write_op.cc:276] Timestamp: P: 1765486510565730 usec, L: 0
1211 20:55:14.680295 (+ 2137us) write_op.cc:270] Start()
1211 20:55:14.680354 (+ 59us) write_op.cc:276] Timestamp: P: 1765486510566351 usec, L: 0
1211 20:55:14.680537 (+ 183us) write_op.cc:270] Start()
1211 20:55:14.680558 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510566963 usec, L: 0
1211 20:55:14.680730 (+ 172us) write_op.cc:270] Start()
1211 20:55:14.680747 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510605333 usec, L: 0
1211 20:55:14.680902 (+ 155us) write_op.cc:270] Start()
1211 20:55:14.680920 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510611837 usec, L: 0
1211 20:55:14.681071 (+ 151us) write_op.cc:270] Start()
1211 20:55:14.681088 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510662434 usec, L: 0
1211 20:55:14.681262 (+ 174us) write_op.cc:270] Start()
1211 20:55:14.681314 (+ 52us) write_op.cc:276] Timestamp: P: 1765486510689661 usec, L: 0
1211 20:55:14.681490 (+ 176us) write_op.cc:270] Start()
1211 20:55:14.681508 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510698135 usec, L: 0
1211 20:55:14.681657 (+ 149us) write_op.cc:270] Start()
1211 20:55:14.681674 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510708821 usec, L: 0
1211 20:55:14.681877 (+ 203us) write_op.cc:270] Start()
1211 20:55:14.681896 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510743440 usec, L: 0
1211 20:55:14.682097 (+ 201us) write_op.cc:270] Start()
1211 20:55:14.682123 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510744494 usec, L: 0
1211 20:55:14.693112 (+ 10989us) write_op.cc:270] Start()
1211 20:55:14.693147 (+ 35us) write_op.cc:276] Timestamp: P: 1765486510747586 usec, L: 0
1211 20:55:14.696074 (+ 2927us) write_op.cc:270] Start()
1211 20:55:14.696095 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510779012 usec, L: 0
1211 20:55:14.696262 (+ 167us) write_op.cc:270] Start()
1211 20:55:14.696279 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510792429 usec, L: 0
1211 20:55:14.702183 (+ 5904us) write_op.cc:270] Start()
1211 20:55:14.702213 (+ 30us) write_op.cc:276] Timestamp: P: 1765486510817191 usec, L: 0
1211 20:55:14.702482 (+ 269us) write_op.cc:270] Start()
1211 20:55:14.702510 (+ 28us) write_op.cc:276] Timestamp: P: 1765486510820544 usec, L: 0
1211 20:55:14.704185 (+ 1675us) write_op.cc:270] Start()
1211 20:55:14.704212 (+ 27us) write_op.cc:276] Timestamp: P: 1765486510823054 usec, L: 0
1211 20:55:14.704396 (+ 184us) write_op.cc:270] Start()
1211 20:55:14.704418 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510836379 usec, L: 0
1211 20:55:14.705501 (+ 1083us) write_op.cc:270] Start()
1211 20:55:14.705523 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510890323 usec, L: 0
1211 20:55:14.705674 (+ 151us) write_op.cc:270] Start()
1211 20:55:14.705695 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510962334 usec, L: 0
1211 20:55:14.715163 (+ 9468us) write_op.cc:270] Start()
1211 20:55:14.715186 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510992806 usec, L: 0
1211 20:55:14.742518 (+ 27332us) write_op.cc:270] Start()
1211 20:55:14.742544 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511001445 usec, L: 0
1211 20:55:14.742733 (+ 189us) write_op.cc:270] Start()
1211 20:55:14.742756 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511009013 usec, L: 0
1211 20:55:14.747227 (+ 4471us) write_op.cc:270] Start()
1211 20:55:14.747253 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511051992 usec, L: 0
1211 20:55:14.758893 (+ 11640us) write_op.cc:270] Start()
1211 20:55:14.758917 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511073398 usec, L: 0
1211 20:55:14.762077 (+ 3160us) write_op.cc:270] Start()
1211 20:55:14.762099 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511096695 usec, L: 0
1211 20:55:14.762299 (+ 200us) write_op.cc:270] Start()
1211 20:55:14.762319 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511116727 usec, L: 0
1211 20:55:14.762476 (+ 157us) write_op.cc:270] Start()
1211 20:55:14.762500 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511120662 usec, L: 0
1211 20:55:14.762653 (+ 153us) write_op.cc:270] Start()
1211 20:55:14.762670 (+ 17us) write_op.cc:276] Timestamp: P: 1765486511121229 usec, L: 0
1211 20:55:14.762798 (+ 128us) write_op.cc:270] Start()
1211 20:55:14.762834 (+ 36us) write_op.cc:276] Timestamp: P: 1765486511154554 usec, L: 0
1211 20:55:14.762979 (+ 145us) write_op.cc:270] Start()
1211 20:55:14.762999 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511199232 usec, L: 0
1211 20:55:14.763162 (+ 163us) write_op.cc:270] Start()
1211 20:55:14.763203 (+ 41us) write_op.cc:276] Timestamp: P: 1765486511210721 usec, L: 0
1211 20:55:14.763340 (+ 137us) write_op.cc:270] Start()
1211 20:55:14.763361 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511227346 usec, L: 0
1211 20:55:14.767277 (+ 3916us) write_op.cc:270] Start()
1211 20:55:14.767305 (+ 28us) write_op.cc:276] Timestamp: P: 1765486511236584 usec, L: 0
1211 20:55:14.799467 (+ 32162us) write_op.cc:270] Start()
1211 20:55:14.799495 (+ 28us) write_op.cc:276] Timestamp: P: 1765486511274627 usec, L: 0
1211 20:55:14.802073 (+ 2578us) write_op.cc:270] Start()
1211 20:55:14.802102 (+ 29us) write_op.cc:276] Timestamp: P: 1765486511280923 usec, L: 0
1211 20:55:14.813672 (+ 11570us) write_op.cc:270] Start()
1211 20:55:14.813697 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511300599 usec, L: 0
1211 20:55:14.819409 (+ 5712us) write_op.cc:270] Start()
1211 20:55:14.819435 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511331522 usec, L: 0
1211 20:55:14.820631 (+ 1196us) write_op.cc:270] Start()
1211 20:55:14.820654 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511352275 usec, L: 0
1211 20:55:14.824065 (+ 3411us) write_op.cc:270] Start()
1211 20:55:14.824092 (+ 27us) write_op.cc:276] Timestamp: P: 1765486511362651 usec, L: 0
1211 20:55:14.825347 (+ 1255us) write_op.cc:270] Start()
1211 20:55:14.825372 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511391746 usec, L: 0
1211 20:55:14.825531 (+ 159us) write_op.cc:270] Start()
1211 20:55:14.825551 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511392325 usec, L: 0
1211 20:55:14.825736 (+ 185us) write_op.cc:270] Start()
1211 20:55:14.825757 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511392956 usec, L: 0
1211 20:55:14.825922 (+ 165us) write_op.cc:270] Start()
1211 20:55:14.825944 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511460009 usec, L: 0
1211 20:55:14.831246 (+ 5302us) write_op.cc:270] Start()
1211 20:55:14.831266 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511482616 usec, L: 0
1211 20:55:14.831418 (+ 152us) write_op.cc:270] Start()
1211 20:55:14.831434 (+ 16us) write_op.cc:276] Timestamp: P: 1765486511494117 usec, L: 0
1211 20:55:14.831579 (+ 145us) write_op.cc:270] Start()
1211 20:55:14.831602 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511503921 usec, L: 0
1211 20:55:14.831751 (+ 149us) write_op.cc:270] Start()
1211 20:55:14.831775 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511534710 usec, L: 0
1211 20:55:14.831931 (+ 156us) write_op.cc:270] Start()
1211 20:55:14.831946 (+ 15us) write_op.cc:276] Timestamp: P: 1765486511547114 usec, L: 0
1211 20:55:14.837665 (+ 5719us) write_op.cc:270] Start()
1211 20:55:14.837687 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511577228 usec, L: 0
1211 20:55:14.837891 (+ 204us) write_op.cc:270] Start()
1211 20:55:14.837914 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511583389 usec, L: 0
1211 20:55:14.838141 (+ 227us) write_op.cc:270] Start()
1211 20:55:14.838163 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511584857 usec, L: 0
1211 20:55:14.839048 (+ 885us) write_op.cc:270] Start()
1211 20:55:14.839084 (+ 36us) write_op.cc:276] Timestamp: P: 1765486511617999 usec, L: 0
1211 20:55:14.839271 (+ 187us) write_op.cc:270] Start()
1211 20:55:14.839294 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511649995 usec, L: 0
1211 20:55:14.839472 (+ 178us) write_op.cc:270] Start()
1211 20:55:14.839492 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511669838 usec, L: 0
1211 20:55:14.839637 (+ 145us) write_op.cc:270] Start()
1211 20:55:14.839656 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511671004 usec, L: 0
1211 20:55:14.839798 (+ 142us) write_op.cc:270] Start()
1211 20:55:14.839816 (+ 18us) write_op.cc:276] Timestamp: P: 1765486511671563 usec, L: 0
1211 20:55:14.839986 (+ 170us) write_op.cc:270] Start()
1211 20:55:14.840005 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511710423 usec, L: 0
1211 20:55:14.840740 (+ 735us) write_op.cc:270] Start()
1211 20:55:14.840764 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511742451 usec, L: 0
1211 20:55:14.840964 (+ 200us) write_op.cc:270] Start()
1211 20:55:14.840985 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511759636 usec, L: 0
1211 20:55:14.841210 (+ 225us) write_op.cc:270] Start()
1211 20:55:14.841233 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511764051 usec, L: 0
1211 20:55:14.841450 (+ 217us) write_op.cc:270] Start()
1211 20:55:14.841470 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511771128 usec, L: 0
1211 20:55:14.845854 (+ 4384us) write_op.cc:270] Start()
1211 20:55:14.845877 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511802995 usec, L: 0
1211 20:55:14.847673 (+ 1796us) write_op.cc:270] Start()
1211 20:55:14.849256 (+ 1583us) write_op.cc:276] Timestamp: P: 1765486511814232 usec, L: 0
1211 20:55:14.849417 (+ 161us) write_op.cc:270] Start()
1211 20:55:14.849437 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511826905 usec, L: 0
1211 20:55:14.855241 (+ 5804us) write_op.cc:270] Start()
1211 20:55:14.855262 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511842737 usec, L: 0
1211 20:55:14.857074 (+ 1812us) write_op.cc:270] Start()
1211 20:55:14.857111 (+ 37us) write_op.cc:276] Timestamp: P: 1765486511856953 usec, L: 0
1211 20:55:14.862582 (+ 5471us) write_op.cc:270] Start()
1211 20:55:14.862605 (+ 23us) write_op.cc:276] Timestamp: P: 176548
I20251211 20:55:16.400158 7051 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:16.430276 7051 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:16.434047 6944 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.194:47609 (request call id 16) took 2186 ms. Trace:
I20251211 20:55:16.440481 7051 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:16.443518 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1204 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:16.444368 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1204 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:16.456282 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:16.456972 7051 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:14.749272 6945 rpcz_store.cc:276] 1211 20:55:10.211735 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:10.212162 (+ 427us) service_pool.cc:225] Handling call
1211 20:55:10.386180 (+174018us) raft_consensus.cc:1471] Updating replica for 826 ops
1211 20:55:10.685938 (+299758us) raft_consensus.cc:1534] Early marking committed up to index 160
1211 20:55:10.721831 (+ 35893us) raft_consensus.cc:1539] Triggering prepare for 826 ops
1211 20:55:10.734286 (+ 12455us) write_op.cc:270] Start()
1211 20:55:10.734316 (+ 30us) write_op.cc:276] Timestamp: P: 1765486495510731 usec, L: 0
1211 20:55:10.734562 (+ 246us) write_op.cc:270] Start()
1211 20:55:10.734604 (+ 42us) write_op.cc:276] Timestamp: P: 1765486495540049 usec, L: 0
1211 20:55:10.734772 (+ 168us) write_op.cc:270] Start()
1211 20:55:10.734804 (+ 32us) write_op.cc:276] Timestamp: P: 1765486495547876 usec, L: 0
1211 20:55:10.735013 (+ 209us) write_op.cc:270] Start()
1211 20:55:10.735033 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495563393 usec, L: 0
1211 20:55:10.735201 (+ 168us) write_op.cc:270] Start()
1211 20:55:10.735224 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495569085 usec, L: 0
1211 20:55:10.735366 (+ 142us) write_op.cc:270] Start()
1211 20:55:10.735389 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495583888 usec, L: 0
1211 20:55:10.735542 (+ 153us) write_op.cc:270] Start()
1211 20:55:10.735583 (+ 41us) write_op.cc:276] Timestamp: P: 1765486495590594 usec, L: 0
1211 20:55:10.735725 (+ 142us) write_op.cc:270] Start()
1211 20:55:10.735744 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495619491 usec, L: 0
1211 20:55:10.735880 (+ 136us) write_op.cc:270] Start()
1211 20:55:10.735899 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495627981 usec, L: 0
1211 20:55:10.736039 (+ 140us) write_op.cc:270] Start()
1211 20:55:10.736058 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495642294 usec, L: 0
1211 20:55:10.736257 (+ 199us) write_op.cc:270] Start()
1211 20:55:10.736278 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495661052 usec, L: 0
1211 20:55:10.736416 (+ 138us) write_op.cc:270] Start()
1211 20:55:10.736434 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495677442 usec, L: 0
1211 20:55:10.752972 (+ 16538us) write_op.cc:270] Start()
1211 20:55:10.753003 (+ 31us) write_op.cc:276] Timestamp: P: 1765486495681638 usec, L: 0
1211 20:55:10.753228 (+ 225us) write_op.cc:270] Start()
1211 20:55:10.753251 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495691100 usec, L: 0
1211 20:55:10.753450 (+ 199us) write_op.cc:270] Start()
1211 20:55:10.753474 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495722438 usec, L: 0
1211 20:55:10.753651 (+ 177us) write_op.cc:270] Start()
1211 20:55:10.753685 (+ 34us) write_op.cc:276] Timestamp: P: 1765486495727197 usec, L: 0
1211 20:55:10.753850 (+ 165us) write_op.cc:270] Start()
1211 20:55:10.753874 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495756293 usec, L: 0
1211 20:55:10.754057 (+ 183us) write_op.cc:270] Start()
1211 20:55:10.754079 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495764823 usec, L: 0
1211 20:55:10.754242 (+ 163us) write_op.cc:270] Start()
1211 20:55:10.754267 (+ 25us) write_op.cc:276] Timestamp: P: 1765486495771084 usec, L: 0
1211 20:55:10.756951 (+ 2684us) write_op.cc:270] Start()
1211 20:55:10.756988 (+ 37us) write_op.cc:276] Timestamp: P: 1765486495800311 usec, L: 0
1211 20:55:10.763757 (+ 6769us) write_op.cc:270] Start()
1211 20:55:10.763788 (+ 31us) write_op.cc:276] Timestamp: P: 1765486495805386 usec, L: 0
1211 20:55:10.771265 (+ 7477us) write_op.cc:270] Start()
1211 20:55:10.771291 (+ 26us) write_op.cc:276] Timestamp: P: 1765486495839206 usec, L: 0
1211 20:55:10.805851 (+ 34560us) write_op.cc:270] Start()
1211 20:55:10.805880 (+ 29us) write_op.cc:276] Timestamp: P: 1765486495860639 usec, L: 0
1211 20:55:10.806102 (+ 222us) write_op.cc:270] Start()
1211 20:55:10.806124 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495882189 usec, L: 0
1211 20:55:10.830220 (+ 24096us) write_op.cc:270] Start()
1211 20:55:10.830253 (+ 33us) write_op.cc:276] Timestamp: P: 1765486495889592 usec, L: 0
1211 20:55:10.841673 (+ 11420us) write_op.cc:270] Start()
1211 20:55:10.841703 (+ 30us) write_op.cc:276] Timestamp: P: 1765486495901710 usec, L: 0
1211 20:55:10.845881 (+ 4178us) write_op.cc:270] Start()
1211 20:55:10.845919 (+ 38us) write_op.cc:276] Timestamp: P: 1765486495912328 usec, L: 0
1211 20:55:10.846129 (+ 210us) write_op.cc:270] Start()
1211 20:55:10.846152 (+ 23us) write_op.cc:276] Timestamp: P: 1765486495932861 usec, L: 0
1211 20:55:10.846345 (+ 193us) write_op.cc:270] Start()
1211 20:55:10.846372 (+ 27us) write_op.cc:276] Timestamp: P: 1765486495949366 usec, L: 0
1211 20:55:10.846536 (+ 164us) write_op.cc:270] Start()
1211 20:55:10.846562 (+ 26us) write_op.cc:276] Timestamp: P: 1765486495960288 usec, L: 0
1211 20:55:10.846720 (+ 158us) write_op.cc:270] Start()
1211 20:55:10.846744 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495965095 usec, L: 0
1211 20:55:10.846955 (+ 211us) write_op.cc:270] Start()
1211 20:55:10.846979 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495967841 usec, L: 0
1211 20:55:10.871422 (+ 24443us) write_op.cc:270] Start()
1211 20:55:10.871462 (+ 40us) write_op.cc:276] Timestamp: P: 1765486496003079 usec, L: 0
1211 20:55:10.882159 (+ 10697us) write_op.cc:270] Start()
1211 20:55:10.882190 (+ 31us) write_op.cc:276] Timestamp: P: 1765486496007086 usec, L: 0
1211 20:55:10.887126 (+ 4936us) write_op.cc:270] Start()
1211 20:55:10.887157 (+ 31us) write_op.cc:276] Timestamp: P: 1765486496021025 usec, L: 0
1211 20:55:10.887374 (+ 217us) write_op.cc:270] Start()
1211 20:55:10.887409 (+ 35us) write_op.cc:276] Timestamp: P: 1765486496031666 usec, L: 0
1211 20:55:10.887579 (+ 170us) write_op.cc:270] Start()
1211 20:55:10.887598 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496058084 usec, L: 0
1211 20:55:10.887745 (+ 147us) write_op.cc:270] Start()
1211 20:55:10.887764 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496060346 usec, L: 0
1211 20:55:10.887895 (+ 131us) write_op.cc:270] Start()
1211 20:55:10.887913 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496075076 usec, L: 0
1211 20:55:10.888072 (+ 159us) write_op.cc:270] Start()
1211 20:55:10.888101 (+ 29us) write_op.cc:276] Timestamp: P: 1765486496079917 usec, L: 0
1211 20:55:10.888255 (+ 154us) write_op.cc:270] Start()
1211 20:55:10.888273 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496105469 usec, L: 0
1211 20:55:10.888424 (+ 151us) write_op.cc:270] Start()
1211 20:55:10.888443 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496114548 usec, L: 0
1211 20:55:10.888573 (+ 130us) write_op.cc:270] Start()
1211 20:55:10.888591 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496116020 usec, L: 0
1211 20:55:10.888725 (+ 134us) write_op.cc:270] Start()
1211 20:55:10.888743 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496140872 usec, L: 0
1211 20:55:10.888876 (+ 133us) write_op.cc:270] Start()
1211 20:55:10.888894 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496145755 usec, L: 0
1211 20:55:10.889047 (+ 153us) write_op.cc:270] Start()
1211 20:55:10.889068 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496153916 usec, L: 0
1211 20:55:10.889220 (+ 152us) write_op.cc:270] Start()
1211 20:55:10.889239 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496157943 usec, L: 0
1211 20:55:10.889402 (+ 163us) write_op.cc:270] Start()
1211 20:55:10.889421 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496187680 usec, L: 0
1211 20:55:10.889552 (+ 131us) write_op.cc:270] Start()
1211 20:55:10.889572 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496189334 usec, L: 0
1211 20:55:10.890200 (+ 628us) write_op.cc:270] Start()
1211 20:55:10.890222 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496202857 usec, L: 0
1211 20:55:10.890413 (+ 191us) write_op.cc:270] Start()
1211 20:55:10.890435 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496220441 usec, L: 0
1211 20:55:10.890581 (+ 146us) write_op.cc:270] Start()
1211 20:55:10.890618 (+ 37us) write_op.cc:276] Timestamp: P: 1765486496239172 usec, L: 0
1211 20:55:10.890758 (+ 140us) write_op.cc:270] Start()
1211 20:55:10.890777 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496251727 usec, L: 0
1211 20:55:10.890912 (+ 135us) write_op.cc:270] Start()
1211 20:55:10.890941 (+ 29us) write_op.cc:276] Timestamp: P: 1765486496252673 usec, L: 0
1211 20:55:10.891069 (+ 128us) write_op.cc:270] Start()
1211 20:55:10.891086 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496281609 usec, L: 0
1211 20:55:10.891253 (+ 167us) write_op.cc:270] Start()
1211 20:55:10.891274 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496284495 usec, L: 0
1211 20:55:10.891470 (+ 196us) write_op.cc:270] Start()
1211 20:55:10.891491 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496293611 usec, L: 0
1211 20:55:10.891652 (+ 161us) write_op.cc:270] Start()
1211 20:55:10.891675 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496315830 usec, L: 0
1211 20:55:10.891816 (+ 141us) write_op.cc:270] Start()
1211 20:55:10.891833 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496336128 usec, L: 0
1211 20:55:10.926151 (+ 34318us) write_op.cc:270] Start()
1211 20:55:10.926215 (+ 64us) write_op.cc:276] Timestamp: P: 1765486496353827 usec, L: 0
1211 20:55:10.926430 (+ 215us) write_op.cc:270] Start()
1211 20:55:10.926451 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496357173 usec, L: 0
1211 20:55:10.931912 (+ 5461us) write_op.cc:270] Start()
1211 20:55:10.931940 (+ 28us) write_op.cc:276] Timestamp: P: 1765486496378498 usec, L: 0
1211 20:55:10.933800 (+ 1860us) write_op.cc:270] Start()
1211 20:55:10.933827 (+ 27us) write_op.cc:276] Timestamp: P: 1765486496394421 usec, L: 0
1211 20:55:10.936868 (+ 3041us) write_op.cc:270] Start()
1211 20:55:10.936892 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496396382 usec, L: 0
1211 20:55:10.953433 (+ 16541us) write_op.cc:270] Start()
1211 20:55:10.953472 (+ 39us) write_op.cc:276] Timestamp: P: 1765486496408173 usec, L: 0
1211 20:55:10.957977 (+ 4505us) write_op.cc:270] Start()
1211 20:55:10.958038 (+ 61us) write_op.cc:276] Timestamp: P: 1765486496418771 usec, L: 0
1211 20:55:10.959557 (+ 1519us) write_op.cc:270] Start()
1211 20:55:10.959593 (+ 36us) write_op.cc:276] Timestamp: P: 1765486496438933 usec, L: 0
1211 20:55:10.961335 (+ 1742us) write_op.cc:270] Start()
1211 20:55:10.961369 (+ 34us) write_op.cc:276] Timestamp: P: 1765486496451797 usec, L: 0
1211 20:55:10.979195 (+ 17826us) write_op.cc:270] Start()
1211 20:55:10.979232 (+ 37us) write_op.cc:276] Timestamp: P: 1765486496470293 usec, L: 0
1211 20:55:10.986444 (+ 7212us) write_op.cc:270] Start()
1211 20:55:10.986475 (+ 31us) write_op.cc:276] Timestamp: P: 1765486496472349 usec, L: 0
1211 20:55:10.986664 (+ 189us) write_op.cc:270] Start()
1211 20:55:10.986686 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496488986 usec, L: 0
1211 20:55:10.989380 (+ 2694us) write_op.cc:270] Start()
1211 20:55:10.989406 (+ 26us) write_op.cc:276] Timestamp: P: 1765486496490372 usec, L: 0
1211 20:55:10.995640 (+ 6234us) write_op.cc:270] Start()
1211 20:55:10.995669 (+ 29us) write_op.cc:276] Timestamp: P: 1765486496506898 usec, L: 0
1211 20:55:10.995862 (+ 193us) write_op.cc:270] Start()
1211 20:55:10.995882 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496522555 usec, L: 0
1211 20:55:10.996052 (+ 170us) write_op.cc:270] Start()
1211 20:55:10.996074 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496528586 usec, L: 0
1211 20:55:10.996236 (+ 162us) write_op.cc:270] Start()
1211 20:55:10.996259 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496537647 usec, L: 0
1211 20:55:11.006970 (+ 10711us) write_op.cc:270] Start()
1211 20:55:11.007012 (+ 42us) write_op.cc:276] Timestamp: P: 1765486496559919 usec, L: 0
1211 20:55:11.009307 (+ 2295us) write_op.cc:270] Start()
1211 20:55:11.011669 (+ 2362us) write_op.cc:276] Timestamp: P: 1765486496571634 usec, L: 0
1211 20:55:11.011872 (+ 203us) write_op.cc:270] Start()
1211 20:55:11.011891 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496577111 usec, L: 0
1211 20:55:11.012054 (+ 163us) write_op.cc:270] Start()
1211 20:55:11.012074 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496582881 usec, L: 0
1211 20:55:11.012211 (+ 137us) write_op.cc:270] Start()
1211 20:55:11.012231 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496607579 usec, L: 0
1211 20:55:11.012422 (+ 191us) write_op.cc:270] Start()
1211 20:55:11.012465 (+ 43us) write_op.cc:276] Timestamp: P: 1765486496624764 usec, L: 0
1211 20:55:11.012617 (+ 152us) write_op.cc:270] Start()
1211 20:55:11.012634 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496628036 usec, L: 0
1211 20:55:11.012750 (+ 116us) write_op.cc:270] Start()
1211 20:55:11.012767 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496639400 usec, L: 0
1211 20:55:11.012891 (+ 124us) write_op.cc:270] Start()
1211 20:55:11.012908 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496644472 usec, L: 0
1211 20:55:11.013015 (+ 107us) write_op.cc:270] Start()
1211 20:55:11.013031 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496671482 usec, L: 0
1211 20:55:11.013196 (+ 165us) write_op.cc:270] Start()
1211 20:55:11.013213 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496693742 usec, L: 0
1211 20:55:11.013353 (+ 140us) write_op.cc:270] Start()
1211 20:55:11.013377 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496696135 usec, L: 0
1211 20:55:11.013517 (+ 140us) write_op.cc:270] Start()
1211 20:55:11.013535 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496698094 usec, L: 0
1211 20:55:11.013662 (+ 127us) write_op.cc:270] Start()
1211 20:55:11.013688 (+ 26us) write_op.cc:276] Timestamp: P: 1765486496723461 usec, L: 0
1211 20:55:11.013823 (+ 135us) write_op.cc:270] Start()
1211 20:55:11.013848 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496738961 usec, L: 0
1211 20:55:11.013988 (+ 140us) write_op.cc:270] Start()
1211 20:55:11.014007 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496747518 usec, L: 0
1211 20:55:11.025763 (+ 11756us) write_op.cc:270] Start()
1211 20:55:11.025787 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496758226 usec, L: 0
1211 20:55:11.025933 (+ 146us) write_op.cc:270] Start()
1211 20:55:11.025952 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496782321 usec, L: 0
1211 20:55:11.035197 (+ 9245us) write_op.cc:270] Start()
1211 20:55:11.035220 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496791416 usec, L: 0
1211 20:55:11.051945 (+ 16725us) write_op.cc:270] Start()
1211 20:55:11.051969 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496799887 usec, L: 0
1211 20:55:11.052139 (+ 170us) write_op.cc:270] Start()
1211 20:55:11.052158 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496815879 usec, L: 0
1211 20:55:11.053839 (+ 1681us) write_op.cc:270] Start()
1211 20:55:11.053859 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496821164 usec, L: 0
1211 20:55:11.066168 (+ 12309us) write_op.cc:270] Start()
1211 20:55:11.066193 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496848405 usec, L: 0
1211 20:55:11.066355 (+ 162us) write_op.cc:270] Start()
1211 20:55:11.066376 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496870231 usec, L: 0
1211 20:55:11.066513 (+ 137us) write_op.cc:270] Start()
1211 20:55:11.066533 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496880867 usec, L: 0
1211 20:55:11.080587 (+ 14054us) write_op.cc:270] Start()
1211 20:55:11.080612 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496901961 usec, L: 0
1211 20:55:11.080782 (+ 170us) write_op.cc:270] Start()
1211 20:55:11.080801 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496909361 usec, L: 0
1211 20:55:11.087999 (+ 7198us) write_op.cc:270] Start()
1211 20:55:11.088023 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496927195 usec, L: 0
1211 20:55:11.095382 (+ 7359us) write_op.cc:270] Start()
1211 20:55:11.095413 (+ 31us) write_op.cc:276] Timestamp: P: 1765486496937289 usec, L: 0
1211 20:55:11.097978 (+ 2565us) write_op.cc:270] Start()
1211 20:55:11.098027 (+ 49us) write_op.cc:276] Timestamp: P: 1765486496967070 usec, L: 0
1211 20:55:11.103816 (+ 5789us) write_op.cc:270] Start()
1211 20:55:11.103838 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496970452 usec, L: 0
1211 20:55:11.106159 (+ 2321us) write_op.cc:270] Start()
1211 20:55:11.106181 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496994745 usec, L: 0
1211 20:55:11.115301 (+ 9120us) write_op.cc:270] Start()
1211 20:55:11.115324 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496995733 usec, L: 0
1211 20:55:11.140761 (+ 25437us) write_op.cc:270] Start()
1211 20:55:11.140783 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497009044 usec, L: 0
1211 20:55:11.140953 (+ 170us) write_op.cc:270] Start()
1211 20:55:11.144999 (+ 4046us) write_op.cc:276] Timestamp: P: 1765486497025245 usec, L: 0
1211 20:55:11.145226 (+ 227us) write_op.cc:270] Start()
1211 20:55:11.145251 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497039040 usec, L: 0
1211 20:55:11.145442 (+ 191us) write_op.cc:270] Start()
1211 20:55:11.145464 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497052031 usec, L: 0
1211 20:55:11.145617 (+ 153us) write_op.cc:270] Start()
1211 20:55:11.145636 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497080923 usec, L: 0
1211 20:55:11.145782 (+ 146us) write_op.cc:270] Start()
1211 20:55:11.145801 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497083825 usec, L: 0
1211 20:55:11.145949 (+ 148us) write_op.cc:270] Start()
1211 20:55:11.145967 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497087300 usec, L: 0
1211 20:55:11.146176 (+ 209us) write_op.cc:270] Start()
1211 20:55:11.146200 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497087932 usec, L: 0
1211 20:55:11.146368 (+ 168us) write_op.cc:270] Start()
1211 20:55:11.146389 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497120168 usec, L: 0
1211 20:55:11.146568 (+ 179us) write_op.cc:270] Start()
1211 20:55:11.146586 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497134545 usec, L: 0
1211 20:55:11.146753 (+ 167us) write_op.cc:270] Start()
1211 20:55:11.146772 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497139522 usec, L: 0
1211 20:55:11.146929 (+ 157us) write_op.cc:270] Start()
1211 20:55:11.146966 (+ 37us) write_op.cc:276] Timestamp: P: 1765486497140195 usec, L: 0
1211 20:55:11.147155 (+ 189us) write_op.cc:270] Start()
1211 20:55:11.147176 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497166324 usec, L: 0
1211 20:55:11.147347 (+ 171us) write_op.cc:270] Start()
1211 20:55:11.147368 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497176573 usec, L: 0
1211 20:55:11.147506 (+ 138us) write_op.cc:270] Start()
1211 20:55:11.147525 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497197605 usec, L: 0
1211 20:55:11.147666 (+ 141us) write_op.cc:270] Start()
1211 20:55:11.147684 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497219169 usec, L: 0
1211 20:55:11.147811 (+ 127us) write_op.cc:270] Start()
1211 20:55:11.147829 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497230300 usec, L: 0
1211 20:55:11.147953 (+ 124us) write_op.cc:270] Start()
1211 20:55:11.147972 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497231260 usec, L: 0
1211 20:55:11.148189 (+ 217us) write_op.cc:270] Start()
1211 20:55:11.148210 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497239383 usec, L: 0
1211 20:55:11.148412 (+ 202us) write_op.cc:270] Start()
1211 20:55:11.148437 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497261854 usec, L: 0
1211 20:55:11.148609 (+ 172us) write_op.cc:270] Start()
1211 20:55:11.148633 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497267045 usec, L: 0
1211 20:55:11.148797 (+ 164us) write_op.cc:270] Start()
1211 20:55:11.181705 (+ 32908us) write_op.cc:276] Timestamp: P: 1765486497298039 usec, L: 0
1211 20:55:11.206071 (+ 24366us) write_op.cc:270] Start()
1211 20:55:11.206101 (+ 30us) write_op.cc:276] Timestamp: P: 1765486497304001 usec, L: 0
1211 20:55:11.206314 (+ 213us) write_op.cc:270] Start()
1211 20:55:11.206337 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497324214 usec, L: 0
1211 20:55:11.206470 (+ 133us) write_op.cc:270] Start()
1211 20:55:11.206493 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497334364 usec, L: 0
1211 20:55:11.206634 (+ 141us) write_op.cc:270] Start()
1211 20:55:11.206654 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497351289 usec, L: 0
1211 20:55:11.206780 (+ 126us) write_op.cc:270] Start()
1211 20:55:11.206797 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497352852 usec, L: 0
1211 20:55:11.206912 (+ 115us) write_op.cc:270] Start()
1211 20:55:11.207042 (+ 130us) write_op.cc:276] Timestamp: P: 1765486497380229 usec, L: 0
1211 20:55:11.207197 (+ 155us) write_op.cc:270] Start()
1211 20:55:11.207214 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497402246 usec, L: 0
1211 20:55:11.207374 (+ 160us) write_op.cc:270] Start()
1211 20:55:11.207392 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497418356 usec, L: 0
1211 20:55:11.207515 (+ 123us) write_op.cc:270] Start()
1211 20:55:11.207531 (+ 16us) write_op.cc:276] Timestamp: P: 1765486497424430 usec, L: 0
1211 20:55:11.209176 (+ 1645us) write_op.cc:270] Start()
1211 20:55:11.209199 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497445669 usec, L: 0
1211 20:55:11.209344 (+ 145us) write_op.cc:270] Start()
1211 20:55:11.209372 (+ 28us) write_op.cc:276] Timestamp: P: 1765486497446508 usec, L: 0
1211 20:55:11.209484 (+ 112us) write_op.cc:270] Start()
1211 20:55:11.225039 (+ 15555us) write_op.cc:276] Timestamp: P: 1765486497453309 usec, L: 0
1211 20:55:11.225239 (+ 200us) write_op.cc:270] Start()
1211 20:55:11.225262 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497454717 usec, L: 0
1211 20:55:11.225423 (+ 161us) write_op.cc:270] Start()
1211 20:55:11.225444 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497490230 usec, L: 0
1211 20:55:11.226652 (+ 1208us) write_op.cc:270] Start()
1211 20:55:11.226675 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497502189 usec, L: 0
1211 20:55:11.226827 (+ 152us) write_op.cc:270] Start()
1211 20:55:11.226847 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497512236 usec, L: 0
1211 20:55:11.227048 (+ 201us) write_op.cc:270] Start()
1211 20:55:11.227072 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497520801 usec, L: 0
1211 20:55:11.227234 (+ 162us) write_op.cc:270] Start()
1211 20:55:11.227256 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497539324 usec, L: 0
1211 20:55:11.227419 (+ 163us) write_op.cc:270] Start()
1211 20:55:11.227438 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497543928 usec, L: 0
1211 20:55:11.227584 (+ 146us) write_op.cc:270] Start()
1211 20:55:11.227601 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497556940 usec, L: 0
1211 20:55:11.227752 (+ 151us) write_op.cc:270] Start()
1211 20:55:11.227776 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497557742 usec, L: 0
1211 20:55:11.229248 (+ 1472us) write_op.cc:270] Start()
1211 20:55:11.229275 (+ 27us) write_op.cc:276] Timestamp: P: 1765486497582468 usec, L: 0
1211 20:55:11.229455 (+ 180us) write_op.cc:270] Start()
1211 20:55:11.229477 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497593237 usec, L: 0
1211 20:55:11.229629 (+ 152us) write_op.cc:270] Start()
1211 20:55:11.229651 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497614664 usec, L: 0
1211 20:55:11.229815 (+ 164us) write_op.cc:270] Start()
1211 20:55:11.229837 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497639730 usec, L: 0
1211 20:55:11.229985 (+ 148us) write_op.cc:270] Start()
1211 20:55:11.230026 (+ 41us) write_op.cc:276] Timestamp: P: 1765486497643009 usec, L: 0
1211 20:55:11.230173 (+ 147us) write_op.cc:270] Start()
1211 20:55:11.230192 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497651266 usec, L: 0
1211 20:55:11.230334 (+ 142us) write_op.cc:270] Start()
1211 20:55:11.231334 (+ 1000us) write_op.cc:276] Timestamp: P: 1765486497666702 usec, L: 0
1211 20:55:11.231503 (+ 169us) write_op.cc:270] Start()
1211 20:55:11.231522 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497683259 usec, L: 0
1211 20:55:11.231647 (+ 125us) write_op.cc:270] Start()
1211 20:55:11.231664 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497689665 usec, L: 0
1211 20:55:11.231799 (+ 135us) write_op.cc:270] Start()
1211 20:55:11.231820 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497706986 usec, L: 0
1211 20:55:11.231965 (+ 145us) write_op.cc:270] Start()
1211 20:55:11.231984 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497726756 usec, L: 0
1211 20:55:11.233798 (+ 1814us) write_op.cc:270] Start()
1211 20:55:11.233823 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497737711 usec, L: 0
1211 20:55:11.233979 (+ 156us) write_op.cc:270] Start()
1211 20:55:11.234019 (+ 40us) write_op.cc:276] Timestamp: P: 1765486497750753 usec, L: 0
1211 20:55:11.234168 (+ 149us) write_op.cc:270] Start()
1211 20:55:11.234825 (+ 657us) write_op.cc:276] Timestamp: P: 1765486497763321 usec, L: 0
1211 20:55:11.235055 (+ 230us) write_op.cc:270] Start()
1211 20:55:11.235076 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497775390 usec, L: 0
1211 20:55:11.236278 (+ 1202us) write_op.cc:270] Start()
1211 20:55:11.236302 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497785428 usec, L: 0
1211 20:55:11.236485 (+ 183us) write_op.cc:270] Start()
1211 20:55:11.236505 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497797652 usec, L: 0
1211 20:55:11.236650 (+ 145us) write_op.cc:270] Start()
1211 20:55:11.236673 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497822798 usec, L: 0
1211 20:55:11.236823 (+ 150us) write_op.cc:270] Start()
1211 20:55:11.236844 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497834612 usec, L: 0
1211 20:55:11.237002 (+ 158us) write_op.cc:270] Start()
1211 20:55:11.237020 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497836771 usec, L: 0
1211 20:55:11.237162 (+ 142us) write_op.cc:270] Start()
1211 20:55:11.237179 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497853316 usec, L: 0
1211 20:55:11.237312 (+ 133us) write_op.cc:270] Start()
1211 20:55:11.237330 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497867193 usec, L: 0
1211 20:55:11.237475 (+ 145us) write_op.cc:270] Start()
1211 20:55:11.237507 (+ 32us) write_op.cc:276] Timestamp: P: 1765486497882702 usec, L: 0
1211 20:55:11.237628 (+ 121us) write_op.cc:270] Start()
1211 20:55:11.237648 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497885081 usec, L: 0
1211 20:55:11.237772 (+ 124us) write_op.cc:270] Start()
1211 20:55:11.237788 (+ 16us) write_op.cc:276] Timestamp: P: 1765486497905186 usec, L: 0
1211 20:55:11.237910 (+ 122us) write_op.cc:270] Start()
1211 20:55:11.237925 (+ 15us) write_op.cc:276] Timestamp: P: 1765486497927930 usec, L: 0
1211 20:55:11.265950 (+ 28025us) write_op.cc:270] Start()
1211 20:55:11.265975 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497944884 usec, L: 0
1211 20:55:11.288659 (+ 22684us) write_op.cc:270] Start()
1211 20:55:11.288685 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497949605 usec, L: 0
1211 20:55:11.288842 (+ 157us) write_op.cc:270] Start()
1211 20:55:11.288860 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497954265 usec, L: 0
1211 20:55:11.290968 (+ 2108us) write_op.cc:270] Start()
1211 20:55:11.290990 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497985864 usec, L: 0
1211 20:55:11.298077 (+ 7087us) write_op.cc:270] Start()
1211 20:55:11.298105 (+ 28us) write_op.cc:276] Timestamp: P: 1765486498013837 usec, L: 0
1211 20:55:11.329617 (+ 31512us) write_op.cc:270] Start()
1211 20:55:11.329649 (+ 32us) write_op.cc:276] Timestamp: P: 1765486498028757 usec, L: 0
1211 20:55:11.349462 (+ 19813us) write_op.cc:270] Start()
1211 20:55:11.349496 (+ 34us) write_op.cc:276] Timestamp: P: 1765486498038368 usec, L: 0
1211 20:55:11.349708 (+ 212us) write_op.cc:270] Start()
1211 20:55:11.349734 (+ 26us) write_op.cc:276] Timestamp: P: 1765486498041690 usec, L: 0
1211 20:55:11.349919 (+ 185us) write_op.cc:270] Start()
1211 20:55:11.349945 (+ 26us) write_op.cc:276] Timestamp: P: 1765486498047240 usec, L: 0
1211 20:55:11.370821 (+ 20876us) write_op.cc:270] Start()
1211 20:55:11.370849 (+ 28us) write_op.cc:276] Timestamp: P: 1765486498076695 usec, L: 0
1211 20:55:11.371019 (+ 170us) write_op.cc:270] Start()
1211 20:55:11.371040 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498088820 usec, L: 0
1211 20:55:11.382641 (+ 11601us) write_op.cc:270] Start()
1211 20:55:11.382669 (+ 28us) write_op.cc:276] Timestamp: P: 1765486498107711 usec, L: 0
1211 20:55:11.389254 (+ 6585us) write_op.cc:270] Start()
1211 20:55:11.393826 (+ 4572us) write_op.cc:276] Timestamp: P: 1765486498132640 usec, L: 0
1211 20:55:11.394057 (+ 231us) write_op.cc:270] Start()
1211 20:55:11.394083 (+ 26us) write_op.cc:276] Timestamp: P: 1765486498134126 usec, L: 0
1211 20:55:11.394260 (+ 177us) write_op.cc:270] Start()
1211 20:55:11.394283 (+ 23us) write_op.cc:276] Timestamp: P: 1765486498136443 usec, L: 0
1211 20:55:11.394460 (+ 177us) write_op.cc:270] Start()
1211 20:55:11.394481 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498161278 usec, L: 0
1211 20:55:11.394641 (+ 160us) write_op.cc:270] Start()
1211 20:55:11.394666 (+ 25us) write_op.cc:276] Timestamp: P: 1765486498181114 usec, L: 0
1211 20:55:11.394837 (+ 171us) write_op.cc:270] Start()
1211 20:55:11.394858 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498185339 usec, L: 0
1211 20:55:11.395006 (+ 148us) write_op.cc:270] Start()
1211 20:55:11.395027 (+ 21us) write_op.cc:276] Timestamp: P: 176548
I20251211 20:55:16.683871 7051 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:16.691301 7051 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:16.692021 7051 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:16.693461 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 986 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:16.697811 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 986 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:16.704574 6872 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:16.706447 7051 raft_consensus.cc:2749] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:16.972575 7066 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:16.972764 7066 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:17.031389 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1204 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:17.032534 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1204 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:17.038619 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:17.040237 7066 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:17.040419 7066 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:55:17.216964 6872 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.211.195:46117, user_credentials={real_user=slave}} blocked reactor thread for 181670us
I20251211 20:55:16.434202 6944 rpcz_store.cc:276] 1211 20:55:14.212692 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:14.219373 (+ 6681us) service_pool.cc:225] Handling call
1211 20:55:14.303855 (+ 84482us) raft_consensus.cc:1471] Updating replica for 323 ops
1211 20:55:14.756488 (+452633us) raft_consensus.cc:1534] Early marking committed up to index 0
1211 20:55:14.756493 (+ 5us) raft_consensus.cc:1539] Triggering prepare for 323 ops
1211 20:55:14.756598 (+ 105us) write_op.cc:270] Start()
1211 20:55:14.756622 (+ 24us) write_op.cc:276] Timestamp: P: 1765486508169389 usec, L: 0
1211 20:55:14.756795 (+ 173us) write_op.cc:270] Start()
1211 20:55:14.756813 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508188862 usec, L: 0
1211 20:55:14.756962 (+ 149us) write_op.cc:270] Start()
1211 20:55:14.756979 (+ 17us) write_op.cc:276] Timestamp: P: 1765486508191865 usec, L: 0
1211 20:55:14.759259 (+ 2280us) write_op.cc:270] Start()
1211 20:55:14.759292 (+ 33us) write_op.cc:276] Timestamp: P: 1765486508199139 usec, L: 0
1211 20:55:14.760696 (+ 1404us) write_op.cc:270] Start()
1211 20:55:14.760714 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508223142 usec, L: 0
1211 20:55:14.760860 (+ 146us) write_op.cc:270] Start()
1211 20:55:14.760875 (+ 15us) write_op.cc:276] Timestamp: P: 1765486508261697 usec, L: 0
1211 20:55:14.763593 (+ 2718us) write_op.cc:270] Start()
1211 20:55:14.774308 (+ 10715us) write_op.cc:276] Timestamp: P: 1765486508264181 usec, L: 0
1211 20:55:14.800149 (+ 25841us) write_op.cc:270] Start()
1211 20:55:14.800170 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508275354 usec, L: 0
1211 20:55:14.800347 (+ 177us) write_op.cc:270] Start()
1211 20:55:14.800372 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508282334 usec, L: 0
1211 20:55:14.800514 (+ 142us) write_op.cc:270] Start()
1211 20:55:14.800530 (+ 16us) write_op.cc:276] Timestamp: P: 1765486508316953 usec, L: 0
1211 20:55:14.800655 (+ 125us) write_op.cc:270] Start()
1211 20:55:14.800676 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508330931 usec, L: 0
1211 20:55:14.823213 (+ 22537us) write_op.cc:270] Start()
1211 20:55:14.823246 (+ 33us) write_op.cc:276] Timestamp: P: 1765486508331473 usec, L: 0
1211 20:55:14.824273 (+ 1027us) write_op.cc:270] Start()
1211 20:55:14.824293 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508349325 usec, L: 0
1211 20:55:14.826129 (+ 1836us) write_op.cc:270] Start()
1211 20:55:14.826151 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508357304 usec, L: 0
1211 20:55:14.828017 (+ 1866us) write_op.cc:270] Start()
1211 20:55:14.828042 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508393837 usec, L: 0
1211 20:55:14.829355 (+ 1313us) write_op.cc:270] Start()
1211 20:55:14.829381 (+ 26us) write_op.cc:276] Timestamp: P: 1765486508394444 usec, L: 0
1211 20:55:14.829564 (+ 183us) write_op.cc:270] Start()
1211 20:55:14.829584 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508395787 usec, L: 0
1211 20:55:14.829744 (+ 160us) write_op.cc:270] Start()
1211 20:55:14.829763 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508401217 usec, L: 0
1211 20:55:14.829956 (+ 193us) write_op.cc:270] Start()
1211 20:55:14.829993 (+ 37us) write_op.cc:276] Timestamp: P: 1765486508446701 usec, L: 0
1211 20:55:14.830214 (+ 221us) write_op.cc:270] Start()
1211 20:55:14.830234 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508474510 usec, L: 0
1211 20:55:14.830447 (+ 213us) write_op.cc:270] Start()
1211 20:55:14.830468 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508475161 usec, L: 0
1211 20:55:14.830605 (+ 137us) write_op.cc:270] Start()
1211 20:55:14.830625 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508497398 usec, L: 0
1211 20:55:14.841428 (+ 10803us) write_op.cc:270] Start()
1211 20:55:14.841453 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508506716 usec, L: 0
1211 20:55:14.843084 (+ 1631us) write_op.cc:270] Start()
1211 20:55:14.843111 (+ 27us) write_op.cc:276] Timestamp: P: 1765486508522067 usec, L: 0
1211 20:55:14.845915 (+ 2804us) write_op.cc:270] Start()
1211 20:55:14.845931 (+ 16us) write_op.cc:276] Timestamp: P: 1765486508529549 usec, L: 0
1211 20:55:14.849038 (+ 3107us) write_op.cc:270] Start()
1211 20:55:14.849063 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508560040 usec, L: 0
1211 20:55:14.850009 (+ 946us) write_op.cc:270] Start()
1211 20:55:14.850034 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508563128 usec, L: 0
1211 20:55:14.852682 (+ 2648us) write_op.cc:270] Start()
1211 20:55:14.852705 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508582185 usec, L: 0
1211 20:55:14.861107 (+ 8402us) write_op.cc:270] Start()
1211 20:55:14.861151 (+ 44us) write_op.cc:276] Timestamp: P: 1765486508614683 usec, L: 0
1211 20:55:14.862582 (+ 1431us) write_op.cc:270] Start()
1211 20:55:14.862605 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508627400 usec, L: 0
1211 20:55:14.870939 (+ 8334us) write_op.cc:270] Start()
1211 20:55:14.870973 (+ 34us) write_op.cc:276] Timestamp: P: 1765486508640186 usec, L: 0
1211 20:55:14.871382 (+ 409us) write_op.cc:270] Start()
1211 20:55:14.871401 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508648999 usec, L: 0
1211 20:55:14.871778 (+ 377us) write_op.cc:270] Start()
1211 20:55:14.871797 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508668655 usec, L: 0
1211 20:55:14.872194 (+ 397us) write_op.cc:270] Start()
1211 20:55:14.872213 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508704275 usec, L: 0
1211 20:55:14.872585 (+ 372us) write_op.cc:270] Start()
1211 20:55:14.872606 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508724410 usec, L: 0
1211 20:55:14.872978 (+ 372us) write_op.cc:270] Start()
1211 20:55:14.872996 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508744914 usec, L: 0
1211 20:55:14.873378 (+ 382us) write_op.cc:270] Start()
1211 20:55:14.873401 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508784143 usec, L: 0
1211 20:55:14.873780 (+ 379us) write_op.cc:270] Start()
1211 20:55:14.873799 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508815512 usec, L: 0
1211 20:55:14.874238 (+ 439us) write_op.cc:270] Start()
1211 20:55:14.874258 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508820373 usec, L: 0
1211 20:55:14.874451 (+ 193us) write_op.cc:270] Start()
1211 20:55:14.874470 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508820984 usec, L: 0
1211 20:55:14.874595 (+ 125us) write_op.cc:270] Start()
1211 20:55:14.874610 (+ 15us) write_op.cc:276] Timestamp: P: 1765486508829916 usec, L: 0
1211 20:55:14.883859 (+ 9249us) write_op.cc:270] Start()
1211 20:55:14.883883 (+ 24us) write_op.cc:276] Timestamp: P: 1765486508897996 usec, L: 0
1211 20:55:14.898378 (+ 14495us) write_op.cc:270] Start()
1211 20:55:14.898405 (+ 27us) write_op.cc:276] Timestamp: P: 1765486508910330 usec, L: 0
1211 20:55:14.898580 (+ 175us) write_op.cc:270] Start()
1211 20:55:14.898604 (+ 24us) write_op.cc:276] Timestamp: P: 1765486508915984 usec, L: 0
1211 20:55:14.899632 (+ 1028us) write_op.cc:270] Start()
1211 20:55:14.899658 (+ 26us) write_op.cc:276] Timestamp: P: 1765486508976575 usec, L: 0
1211 20:55:14.902177 (+ 2519us) write_op.cc:270] Start()
1211 20:55:14.902206 (+ 29us) write_op.cc:276] Timestamp: P: 1765486509005939 usec, L: 0
1211 20:55:14.902399 (+ 193us) write_op.cc:270] Start()
1211 20:55:14.902422 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509008385 usec, L: 0
1211 20:55:14.906593 (+ 4171us) write_op.cc:270] Start()
1211 20:55:14.906617 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509054598 usec, L: 0
1211 20:55:14.916183 (+ 9566us) write_op.cc:270] Start()
1211 20:55:14.916214 (+ 31us) write_op.cc:276] Timestamp: P: 1765486509055159 usec, L: 0
1211 20:55:14.926470 (+ 10256us) write_op.cc:270] Start()
1211 20:55:14.926500 (+ 30us) write_op.cc:276] Timestamp: P: 1765486509090837 usec, L: 0
1211 20:55:14.926682 (+ 182us) write_op.cc:270] Start()
1211 20:55:14.926705 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509099283 usec, L: 0
1211 20:55:14.926854 (+ 149us) write_op.cc:270] Start()
1211 20:55:14.926875 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509124262 usec, L: 0
1211 20:55:14.927064 (+ 189us) write_op.cc:270] Start()
1211 20:55:14.927086 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509138469 usec, L: 0
1211 20:55:14.927246 (+ 160us) write_op.cc:270] Start()
1211 20:55:14.927270 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509167155 usec, L: 0
1211 20:55:14.952216 (+ 24946us) write_op.cc:270] Start()
1211 20:55:14.952239 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509199804 usec, L: 0
1211 20:55:14.965515 (+ 13276us) write_op.cc:270] Start()
1211 20:55:14.965541 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509217896 usec, L: 0
1211 20:55:15.010873 (+ 45332us) write_op.cc:270] Start()
1211 20:55:15.010903 (+ 30us) write_op.cc:276] Timestamp: P: 1765486509240115 usec, L: 0
1211 20:55:15.011121 (+ 218us) write_op.cc:270] Start()
1211 20:55:15.011169 (+ 48us) write_op.cc:276] Timestamp: P: 1765486509254147 usec, L: 0
1211 20:55:15.011340 (+ 171us) write_op.cc:270] Start()
1211 20:55:15.011362 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509260987 usec, L: 0
1211 20:55:15.011519 (+ 157us) write_op.cc:270] Start()
1211 20:55:15.011561 (+ 42us) write_op.cc:276] Timestamp: P: 1765486509300404 usec, L: 0
1211 20:55:15.011712 (+ 151us) write_op.cc:270] Start()
1211 20:55:15.011734 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509304300 usec, L: 0
1211 20:55:15.011886 (+ 152us) write_op.cc:270] Start()
1211 20:55:15.011906 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509325396 usec, L: 0
1211 20:55:15.012058 (+ 152us) write_op.cc:270] Start()
1211 20:55:15.012078 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509336645 usec, L: 0
1211 20:55:15.012261 (+ 183us) write_op.cc:270] Start()
1211 20:55:15.012283 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509351393 usec, L: 0
1211 20:55:15.012441 (+ 158us) write_op.cc:270] Start()
1211 20:55:15.012462 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509376985 usec, L: 0
1211 20:55:15.034516 (+ 22054us) write_op.cc:270] Start()
1211 20:55:15.034549 (+ 33us) write_op.cc:276] Timestamp: P: 1765486509394578 usec, L: 0
1211 20:55:15.036261 (+ 1712us) write_op.cc:270] Start()
1211 20:55:15.036289 (+ 28us) write_op.cc:276] Timestamp: P: 1765486509409087 usec, L: 0
1211 20:55:15.037885 (+ 1596us) write_op.cc:270] Start()
1211 20:55:15.038043 (+ 158us) write_op.cc:276] Timestamp: P: 1765486509430803 usec, L: 0
1211 20:55:15.042449 (+ 4406us) write_op.cc:270] Start()
1211 20:55:15.042469 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509434550 usec, L: 0
1211 20:55:15.042579 (+ 110us) write_op.cc:270] Start()
1211 20:55:15.042592 (+ 13us) write_op.cc:276] Timestamp: P: 1765486509443499 usec, L: 0
1211 20:55:15.045526 (+ 2934us) write_op.cc:270] Start()
1211 20:55:15.045545 (+ 19us) write_op.cc:276] Timestamp: P: 1765486509453142 usec, L: 0
1211 20:55:15.045681 (+ 136us) write_op.cc:270] Start()
1211 20:55:15.045698 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509465879 usec, L: 0
1211 20:55:15.045819 (+ 121us) write_op.cc:270] Start()
1211 20:55:15.045835 (+ 16us) write_op.cc:276] Timestamp: P: 1765486509482895 usec, L: 0
1211 20:55:15.045958 (+ 123us) write_op.cc:270] Start()
1211 20:55:15.045994 (+ 36us) write_op.cc:276] Timestamp: P: 1765486509505338 usec, L: 0
1211 20:55:15.046183 (+ 189us) write_op.cc:270] Start()
1211 20:55:15.046203 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509519018 usec, L: 0
1211 20:55:15.046333 (+ 130us) write_op.cc:270] Start()
1211 20:55:15.046350 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509532019 usec, L: 0
1211 20:55:15.046529 (+ 179us) write_op.cc:270] Start()
1211 20:55:15.046551 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509543543 usec, L: 0
1211 20:55:15.046681 (+ 130us) write_op.cc:270] Start()
1211 20:55:15.046699 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509565994 usec, L: 0
1211 20:55:15.047156 (+ 457us) write_op.cc:270] Start()
1211 20:55:15.047179 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509579643 usec, L: 0
1211 20:55:15.047348 (+ 169us) write_op.cc:270] Start()
1211 20:55:15.047381 (+ 33us) write_op.cc:276] Timestamp: P: 1765486509581929 usec, L: 0
1211 20:55:15.047525 (+ 144us) write_op.cc:270] Start()
1211 20:55:15.047554 (+ 29us) write_op.cc:276] Timestamp: P: 1765486509600010 usec, L: 0
1211 20:55:15.047732 (+ 178us) write_op.cc:270] Start()
1211 20:55:15.047751 (+ 19us) write_op.cc:276] Timestamp: P: 1765486509615985 usec, L: 0
1211 20:55:15.047885 (+ 134us) write_op.cc:270] Start()
1211 20:55:15.047902 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509657733 usec, L: 0
1211 20:55:15.048100 (+ 198us) write_op.cc:270] Start()
1211 20:55:15.048130 (+ 30us) write_op.cc:276] Timestamp: P: 1765486509664888 usec, L: 0
1211 20:55:15.048298 (+ 168us) write_op.cc:270] Start()
1211 20:55:15.048315 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509685700 usec, L: 0
1211 20:55:15.048441 (+ 126us) write_op.cc:270] Start()
1211 20:55:15.048457 (+ 16us) write_op.cc:276] Timestamp: P: 1765486509707524 usec, L: 0
1211 20:55:15.048577 (+ 120us) write_op.cc:270] Start()
1211 20:55:15.048593 (+ 16us) write_op.cc:276] Timestamp: P: 1765486509731636 usec, L: 0
1211 20:55:15.091152 (+ 42559us) write_op.cc:270] Start()
1211 20:55:15.091418 (+ 266us) write_op.cc:276] Timestamp: P: 1765486509734108 usec, L: 0
1211 20:55:15.184462 (+ 93044us) spinlock_profiling.cc:243] Waited 26 ms on lock 0x60f0000180c8. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fdcdc2b8f78 00007fdcdf230dc9 00007fdcdf1d7eb9 00007fdcdf1d56c7 00007fdce304f21d 00007fdcdd1e8b46 00007fdcdd1e8875 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685
1211 20:55:15.184573 (+ 111us) write_op.cc:270] Start()
1211 20:55:15.184597 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509763799 usec, L: 0
1211 20:55:15.184802 (+ 205us) write_op.cc:270] Start()
1211 20:55:15.184823 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509801261 usec, L: 0
1211 20:55:15.186517 (+ 1694us) write_op.cc:270] Start()
1211 20:55:15.186542 (+ 25us) write_op.cc:276] Timestamp: P: 1765486509817085 usec, L: 0
1211 20:55:15.186756 (+ 214us) write_op.cc:270] Start()
1211 20:55:15.186776 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509817680 usec, L: 0
1211 20:55:15.190934 (+ 4158us) write_op.cc:270] Start()
1211 20:55:15.190957 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509849350 usec, L: 0
1211 20:55:15.191094 (+ 137us) write_op.cc:270] Start()
1211 20:55:15.191112 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509873037 usec, L: 0
1211 20:55:15.191260 (+ 148us) write_op.cc:270] Start()
1211 20:55:15.191279 (+ 19us) write_op.cc:276] Timestamp: P: 1765486509894780 usec, L: 0
1211 20:55:15.191410 (+ 131us) write_op.cc:270] Start()
1211 20:55:15.191428 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509913843 usec, L: 0
1211 20:55:15.192724 (+ 1296us) write_op.cc:270] Start()
1211 20:55:15.192746 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509917933 usec, L: 0
1211 20:55:15.192925 (+ 179us) write_op.cc:270] Start()
1211 20:55:15.192946 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509922799 usec, L: 0
1211 20:55:15.193090 (+ 144us) write_op.cc:270] Start()
1211 20:55:15.193233 (+ 143us) write_op.cc:276] Timestamp: P: 1765486509961025 usec, L: 0
1211 20:55:15.193387 (+ 154us) write_op.cc:270] Start()
1211 20:55:15.193408 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509977668 usec, L: 0
1211 20:55:15.193558 (+ 150us) write_op.cc:270] Start()
1211 20:55:15.193576 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509995005 usec, L: 0
1211 20:55:15.195423 (+ 1847us) write_op.cc:270] Start()
1211 20:55:15.195443 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510001373 usec, L: 0
1211 20:55:15.195635 (+ 192us) write_op.cc:270] Start()
1211 20:55:15.195653 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510027843 usec, L: 0
1211 20:55:15.195826 (+ 173us) write_op.cc:270] Start()
1211 20:55:15.195845 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510029552 usec, L: 0
1211 20:55:15.195988 (+ 143us) write_op.cc:270] Start()
1211 20:55:15.196005 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510056643 usec, L: 0
1211 20:55:15.196162 (+ 157us) write_op.cc:270] Start()
1211 20:55:15.196182 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510057317 usec, L: 0
1211 20:55:15.213331 (+ 17149us) write_op.cc:270] Start()
1211 20:55:15.213356 (+ 25us) write_op.cc:276] Timestamp: P: 1765486510109345 usec, L: 0
1211 20:55:15.220314 (+ 6958us) write_op.cc:270] Start()
1211 20:55:15.220334 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510110492 usec, L: 0
1211 20:55:15.223437 (+ 3103us) write_op.cc:270] Start()
1211 20:55:15.223461 (+ 24us) write_op.cc:276] Timestamp: P: 1765486510143590 usec, L: 0
1211 20:55:15.223665 (+ 204us) write_op.cc:270] Start()
1211 20:55:15.223686 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510207563 usec, L: 0
1211 20:55:15.241171 (+ 17485us) write_op.cc:270] Start()
1211 20:55:15.241193 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510218742 usec, L: 0
1211 20:55:15.242906 (+ 1713us) write_op.cc:270] Start()
1211 20:55:15.242931 (+ 25us) write_op.cc:276] Timestamp: P: 1765486510230771 usec, L: 0
1211 20:55:15.253699 (+ 10768us) write_op.cc:270] Start()
1211 20:55:15.253723 (+ 24us) write_op.cc:276] Timestamp: P: 1765486510241330 usec, L: 0
1211 20:55:15.258975 (+ 5252us) write_op.cc:270] Start()
1211 20:55:15.258991 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510280623 usec, L: 0
1211 20:55:15.265231 (+ 6240us) write_op.cc:270] Start()
1211 20:55:15.265259 (+ 28us) write_op.cc:276] Timestamp: P: 1765486510313179 usec, L: 0
1211 20:55:15.266574 (+ 1315us) write_op.cc:270] Start()
1211 20:55:15.266593 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510326541 usec, L: 0
1211 20:55:15.269825 (+ 3232us) write_op.cc:270] Start()
1211 20:55:15.269844 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510361593 usec, L: 0
1211 20:55:15.296030 (+ 26186us) write_op.cc:270] Start()
1211 20:55:15.296047 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510386343 usec, L: 0
1211 20:55:15.296485 (+ 438us) write_op.cc:270] Start()
1211 20:55:15.296505 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510387117 usec, L: 0
1211 20:55:15.296847 (+ 342us) write_op.cc:270] Start()
1211 20:55:15.296866 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510424127 usec, L: 0
1211 20:55:15.297273 (+ 407us) write_op.cc:270] Start()
1211 20:55:15.297291 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510435978 usec, L: 0
1211 20:55:15.297686 (+ 395us) write_op.cc:270] Start()
1211 20:55:15.297706 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510477305 usec, L: 0
1211 20:55:15.298296 (+ 590us) write_op.cc:270] Start()
1211 20:55:15.298317 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510488073 usec, L: 0
1211 20:55:15.298478 (+ 161us) write_op.cc:270] Start()
1211 20:55:15.298497 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510488782 usec, L: 0
1211 20:55:15.298653 (+ 156us) write_op.cc:270] Start()
1211 20:55:15.298673 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510519115 usec, L: 0
1211 20:55:15.298825 (+ 152us) write_op.cc:270] Start()
1211 20:55:15.298844 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510554438 usec, L: 0
1211 20:55:15.298985 (+ 141us) write_op.cc:270] Start()
1211 20:55:15.299002 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510563142 usec, L: 0
1211 20:55:15.299168 (+ 166us) write_op.cc:270] Start()
1211 20:55:15.299213 (+ 45us) write_op.cc:276] Timestamp: P: 1765486510567050 usec, L: 0
1211 20:55:15.299398 (+ 185us) write_op.cc:270] Start()
1211 20:55:15.299430 (+ 32us) write_op.cc:276] Timestamp: P: 1765486510607653 usec, L: 0
1211 20:55:15.299628 (+ 198us) write_op.cc:270] Start()
1211 20:55:15.299650 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510612767 usec, L: 0
1211 20:55:15.299849 (+ 199us) write_op.cc:270] Start()
1211 20:55:15.299871 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510648104 usec, L: 0
1211 20:55:15.300049 (+ 178us) write_op.cc:270] Start()
1211 20:55:15.300071 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510668220 usec, L: 0
1211 20:55:15.300258 (+ 187us) write_op.cc:270] Start()
1211 20:55:15.300279 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510688963 usec, L: 0
1211 20:55:15.300472 (+ 193us) write_op.cc:270] Start()
1211 20:55:15.300493 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510708721 usec, L: 0
1211 20:55:15.300660 (+ 167us) write_op.cc:270] Start()
1211 20:55:15.300677 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510713470 usec, L: 0
1211 20:55:15.300846 (+ 169us) write_op.cc:270] Start()
1211 20:55:15.300864 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510743823 usec, L: 0
1211 20:55:15.301049 (+ 185us) write_op.cc:270] Start()
1211 20:55:15.301067 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510744506 usec, L: 0
1211 20:55:15.301258 (+ 191us) write_op.cc:270] Start()
1211 20:55:15.301277 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510746974 usec, L: 0
1211 20:55:15.301441 (+ 164us) write_op.cc:270] Start()
1211 20:55:15.301460 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510748730 usec, L: 0
1211 20:55:15.301624 (+ 164us) write_op.cc:270] Start()
1211 20:55:15.301643 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510800260 usec, L: 0
1211 20:55:15.301801 (+ 158us) write_op.cc:270] Start()
1211 20:55:15.301819 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510812553 usec, L: 0
1211 20:55:15.306017 (+ 4198us) write_op.cc:270] Start()
1211 20:55:15.306043 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510816591 usec, L: 0
1211 20:55:15.306248 (+ 205us) write_op.cc:270] Start()
1211 20:55:15.306267 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510819933 usec, L: 0
1211 20:55:15.306453 (+ 186us) write_op.cc:270] Start()
1211 20:55:15.306474 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510836405 usec, L: 0
1211 20:55:15.306637 (+ 163us) write_op.cc:270] Start()
1211 20:55:15.306655 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510888565 usec, L: 0
1211 20:55:15.306808 (+ 153us) write_op.cc:270] Start()
1211 20:55:15.306826 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510971365 usec, L: 0
1211 20:55:15.307028 (+ 202us) write_op.cc:270] Start()
1211 20:55:15.307049 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510995709 usec, L: 0
1211 20:55:15.307252 (+ 203us) write_op.cc:270] Start()
1211 20:55:15.307273 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511002701 usec, L: 0
1211 20:55:15.307437 (+ 164us) write_op.cc:270] Start()
1211 20:55:15.307458 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511012234 usec, L: 0
1211 20:55:15.307631 (+ 173us) write_op.cc:270] Start()
1211 20:55:15.307667 (+ 36us) write_op.cc:276] Timestamp: P: 1765486511051292 usec, L: 0
1211 20:55:15.307826 (+ 159us) write_op.cc:270] Start()
1211 20:55:15.307847 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511074206 usec, L: 0
1211 20:55:15.308003 (+ 156us) write_op.cc:270] Start()
1211 20:55:15.308022 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511090086 usec, L: 0
1211 20:55:15.308177 (+ 155us) write_op.cc:270] Start()
1211 20:55:15.308206 (+ 29us) write_op.cc:276] Timestamp: P: 1765486511115386 usec, L: 0
1211 20:55:15.308342 (+ 136us) write_op.cc:270] Start()
1211 20:55:15.308359 (+ 17us) write_op.cc:276] Timestamp: P: 1765486511117696 usec, L: 0
1211 20:55:15.308540 (+ 181us) write_op.cc:270] Start()
1211 20:55:15.308560 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511121784 usec, L: 0
1211 20:55:15.308739 (+ 179us) write_op.cc:270] Start()
1211 20:55:15.308761 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511154629 usec, L: 0
1211 20:55:15.308979 (+ 218us) write_op.cc:270] Start()
1211 20:55:15.309006 (+ 27us) write_op.cc:276] Timestamp: P: 1765486511202044 usec, L: 0
1211 20:55:15.309216 (+ 210us) write_op.cc:270] Start()
1211 20:55:15.309237 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511212876 usec, L: 0
1211 20:55:15.309444 (+ 207us) write_op.cc:270] Start()
1211 20:55:15.309465 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511226691 usec, L: 0
1211 20:55:15.309659 (+ 194us) write_op.cc:270] Start()
1211 20:55:15.309680 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511250465 usec, L: 0
1211 20:55:15.309865 (+ 185us) write_op.cc:270] Start()
1211 20:55:15.309886 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511277204 usec, L: 0
1211 20:55:15.336600 (+ 26714us) write_op.cc:270] Start()
1211 20:55:15.336624 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511281505 usec, L: 0
1211 20:55:15.339318 (+ 2694us) write_op.cc:270] Start()
1211 20:55:15.339342 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511283136 usec, L: 0
1211 20:55:15.352681 (+ 13339us) write_op.cc:270] Start()
1211 20:55:15.352710 (+ 29us) write_op.cc:276] Timestamp: P: 1765486511330786 usec, L: 0
1211 20:55:15.391209 (+ 38499us) write_op.cc:270] Start()
1211 20:55:15.391233 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511351728 usec, L: 0
1211 20:55:15.422781 (+ 31548us) write_op.cc:270] Start()
1211 20:55:15.424344 (+ 1563us) write_op.cc:276] Timestamp: P: 1765486511362460 usec, L: 0
1211 20:55:15.424517 (+ 173us) write_op.cc:270] Start()
1211 20:55:15.424537 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511392251 usec, L: 0
1211 20:55:15.424681 (+ 144us) write_op.cc:270] Start()
1211 20:55:15.424700 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511393004 usec, L: 0
1211 20:55:15.425966 (+ 1266us) write_op.cc:270] Start()
1211 20:55:15.426032 (+ 66us) write_op.cc:276] Timestamp: P: 1765486511394284 usec, L: 0
1211 20:55:15.426214 (+ 182us) write_op.cc:270] Start()
1211 20:55:15.426235 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511463119 usec, L: 0
1211 20:55:15.426401 (+ 166us) write_op.cc:270] Start()
1211 20:55:15.426421 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511483031 usec, L: 0
1211 20:55:15.426587 (+ 166us) write_op.cc:270] Start()
1211 20:55:15.426607 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511496330 usec, L: 0
1211 20:55:15.426806 (+ 199us) write_op.cc:270] Start()
1211 20:55:15.426826 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511505027 usec, L: 0
1211 20:55:15.427049 (+ 223us) write_op.cc:270] Start()
1211 20:55:15.427070 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511534018 usec, L: 0
1211 20:55:15.427248 (+ 178us) write_op.cc:270] Start()
1211 20:55:15.427269 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511550540 usec, L: 0
1211 20:55:15.427468 (+ 199us) write_op.cc:270] Start()
1211 20:55:15.427493 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511574371 usec, L: 0
1211 20:55:15.427692 (+ 199us) write_op.cc:270] Start()
1211 20:55:15.427716 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511578166 usec, L: 0
1211 20:55:15.427872 (+ 156us) write_op.cc:270] Start()
1211 20:55:15.427895 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511578871 usec, L: 0
1211 20:55:15.428077 (+ 182us) write_op.cc:270] Start()
1211 20:55:15.428099 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511620514 usec, L: 0
1211 20:55:15.428288 (+ 189us) write_op.cc:270] Start()
1211 20:55:15.428309 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511642325 usec, L: 0
1211 20:55:15.428472 (+ 163us) write_op.cc:270] Start()
1211 20:55:15.428494 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511649336 usec, L: 0
1211 20:55:15.428685 (+ 191us) write_op.cc:270] Start()
1211 20:55:15.436989 (+ 8304us) write_op.cc:276] Timestamp: P: 1765486511662130 usec, L: 0
1211 20:55:15.437172 (+ 183us) write_op.cc:270] Start()
1211 20:55:15.437192 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511670457 usec, L: 0
1211 20:55:15.446618 (+ 9426us) write_op.cc:270] Start()
1211 20:55:15.446642 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511716557 usec, L: 0
1211 20:55:15.455325 (+ 8683us) write_op.cc:270] Start()
1211 20:55:15.455358 (+ 33us) write_op.cc:276] Timestamp: P: 1765486511741216 usec, L: 0
1211 20:55:15.467816 (+ 12458us) write_op.cc:270] Start()
1211 20:55:15.467842 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511758201 usec, L: 0
1211 20:55:15.477177 (+ 9335us) write_op.cc:270] Start()
1211 20:55:15.477200 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511759715 usec, L: 0
1211 20:55:15.492247 (+ 15047us) write_op.cc:270] Start()
1211 20:55:15.492275 (+ 28us) write_op.cc:276] Timestamp: P: 1765486511772168 usec, L: 0
1211 20:55:15.494921 (+ 2646us) write_op.cc:270] Start()
1211 20:55:15.494943 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511800714 usec, L: 0
1211 20:55:15.495079 (+ 136us) write_op.cc:270] Start()
1211 20:55:15.495103 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511812741 usec, L: 0
1211 20:55:15.495290 (+ 187us) write_op.cc:270] Start()
1211 20:55:15.495311 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511829149 usec, L: 0
1211 20:55:15.495462 (+ 151us) write_op.cc:270] Start()
1211 20:55:15.495481 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511841184 usec, L: 0
1211 20:55:15.495622 (+ 141us) write_op.cc:270] Start()
1211 20:55:15.495640 (+ 18us) write_op.cc:276] Timestamp: P: 1765486511866598 usec, L: 0
1211 20:55:15.495776 (+ 136us) write_op.cc:270] Start()
1211 20:55:15.495793 (+ 17us) write_op.cc:276] Timestamp: P: 1765486511877132 usec, L: 0
1211 20:55:15.497330 (+ 1537us) write_op.cc:270] Star
I20251211 20:55:17.357412 7051 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:17.357607 7051 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:17.358340 7051 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:17.358953 6941 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.194:47609 (request call id 20) took 1052 ms. Trace:
I20251211 20:55:17.362922 7068 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:17.363162 7068 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:17.364436 7068 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:17.450604 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1323 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:17.450704 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1361 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:17.451056 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "d915f836ed2c479c9d6623e11266ed55" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1323 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:17.454627 6901 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:42702 (request call id 8) took 6262 ms. Trace:
I20251211 20:55:17.458848 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "e7ef1dab1f4c47379f2756f162302ecc" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1361 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:17.462280 6946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.195:38469 (request call id 13) took 2368 ms. Trace:
I20251211 20:55:17.454768 6901 rpcz_store.cc:276] 1211 20:55:11.186534 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:11.186598 (+ 64us) service_pool.cc:225] Handling call
1211 20:55:14.735544 (+3548946us) spinlock_profiling.cc:243] Waited 3.54 s on lock 0x61600001a254. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fdcdf1c32c0 00007fdce3073b2f 00007fdcdd498c36 00007fdcdd498965 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685 00007fdcda2cefaf 00007fdcda2cedd8 0000000000a8970c
1211 20:55:15.739130 (+1003586us) spinlock_profiling.cc:243] Waited 995 ms on lock 0x616000019654. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fdcdf1c32c0 00007fdce3073b2f 00007fdcdd498c36 00007fdcdd498965 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685 00007fdcda2cefaf 00007fdcda2cedd8 0000000000a8970c
1211 20:55:15.773004 (+ 33874us) spinlock_profiling.cc:243] Waited 33.5 ms on lock 0x616000019c54. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fdcdf1c32c0 00007fdce3073b2f 00007fdcdd498c36 00007fdcdd498965 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685 00007fdcda2cefaf 00007fdcda2cedd8 0000000000a8970c
1211 20:55:17.448289 (+1675285us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x616000018a54. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fdcdf1c32c0 00007fdce3073b2f 00007fdcdd498c36 00007fdcdd498965 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685 00007fdcda2cefaf 00007fdcda2cedd8 0000000000a8970c
1211 20:55:17.448737 (+ 448us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":14365731200}
I20251211 20:55:17.517797 7057 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:17.523173 6872 leader_election.cc:304] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:17.523696 6872 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:17.566479 7066 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:17.566738 7057 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:17.569443 7057 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:17.573155 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1253 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:17.573195 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1253 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:17.598479 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:17.598974 7066 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:17.599169 7057 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:17.623036 7057 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:17.623819 7057 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:17.635382 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1204 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:17.635821 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1204 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:17.640064 7051 raft_consensus.cc:2749] T e7ef1dab1f4c47379f2756f162302ecc P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:17.657210 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:17.662191 7051 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:17.359442 6941 rpcz_store.cc:276] 1211 20:55:16.306951 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:16.307085 (+ 134us) service_pool.cc:225] Handling call
1211 20:55:16.350681 (+ 43596us) raft_consensus.cc:1471] Updating replica for 143 ops
1211 20:55:16.480587 (+129906us) raft_consensus.cc:1534] Early marking committed up to index 1180
1211 20:55:16.480629 (+ 42us) raft_consensus.cc:1539] Triggering prepare for 143 ops
1211 20:55:16.480747 (+ 118us) write_op.cc:270] Start()
1211 20:55:16.480771 (+ 24us) write_op.cc:276] Timestamp: P: 1765486513708116 usec, L: 0
1211 20:55:16.480941 (+ 170us) write_op.cc:270] Start()
1211 20:55:16.480962 (+ 21us) write_op.cc:276] Timestamp: P: 1765486513755581 usec, L: 0
1211 20:55:16.481104 (+ 142us) write_op.cc:270] Start()
1211 20:55:16.481125 (+ 21us) write_op.cc:276] Timestamp: P: 1765486513756156 usec, L: 0
1211 20:55:16.481290 (+ 165us) write_op.cc:270] Start()
1211 20:55:16.481310 (+ 20us) write_op.cc:276] Timestamp: P: 1765486513763382 usec, L: 0
1211 20:55:16.481503 (+ 193us) write_op.cc:270] Start()
1211 20:55:16.481525 (+ 22us) write_op.cc:276] Timestamp: P: 1765486513779958 usec, L: 0
1211 20:55:16.481670 (+ 145us) write_op.cc:270] Start()
1211 20:55:16.481689 (+ 19us) write_op.cc:276] Timestamp: P: 1765486513805076 usec, L: 0
1211 20:55:16.481830 (+ 141us) write_op.cc:270] Start()
1211 20:55:16.481893 (+ 63us) write_op.cc:276] Timestamp: P: 1765486513807932 usec, L: 0
1211 20:55:16.501918 (+ 20025us) write_op.cc:270] Start()
1211 20:55:16.501949 (+ 31us) write_op.cc:276] Timestamp: P: 1765486513816104 usec, L: 0
1211 20:55:16.502177 (+ 228us) write_op.cc:270] Start()
1211 20:55:16.502202 (+ 25us) write_op.cc:276] Timestamp: P: 1765486513844912 usec, L: 0
1211 20:55:16.502377 (+ 175us) write_op.cc:270] Start()
1211 20:55:16.502480 (+ 103us) write_op.cc:276] Timestamp: P: 1765486513851076 usec, L: 0
1211 20:55:16.502638 (+ 158us) write_op.cc:270] Start()
1211 20:55:16.502662 (+ 24us) write_op.cc:276] Timestamp: P: 1765486513858159 usec, L: 0
1211 20:55:16.502798 (+ 136us) write_op.cc:270] Start()
1211 20:55:16.502822 (+ 24us) write_op.cc:276] Timestamp: P: 1765486513890392 usec, L: 0
1211 20:55:16.502989 (+ 167us) write_op.cc:270] Start()
1211 20:55:16.503028 (+ 39us) write_op.cc:276] Timestamp: P: 1765486513912941 usec, L: 0
1211 20:55:16.503217 (+ 189us) write_op.cc:270] Start()
1211 20:55:16.503240 (+ 23us) write_op.cc:276] Timestamp: P: 1765486513931791 usec, L: 0
1211 20:55:16.503393 (+ 153us) write_op.cc:270] Start()
1211 20:55:16.503416 (+ 23us) write_op.cc:276] Timestamp: P: 1765486513962291 usec, L: 0
1211 20:55:16.503573 (+ 157us) write_op.cc:270] Start()
1211 20:55:16.503597 (+ 24us) write_op.cc:276] Timestamp: P: 1765486513984613 usec, L: 0
1211 20:55:16.503755 (+ 158us) write_op.cc:270] Start()
1211 20:55:16.503778 (+ 23us) write_op.cc:276] Timestamp: P: 1765486514020556 usec, L: 0
1211 20:55:16.504365 (+ 587us) write_op.cc:270] Start()
1211 20:55:16.504390 (+ 25us) write_op.cc:276] Timestamp: P: 1765486514044072 usec, L: 0
1211 20:55:16.504605 (+ 215us) write_op.cc:270] Start()
1211 20:55:16.504627 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514075340 usec, L: 0
1211 20:55:16.504786 (+ 159us) write_op.cc:270] Start()
1211 20:55:16.504815 (+ 29us) write_op.cc:276] Timestamp: P: 1765486514084412 usec, L: 0
1211 20:55:16.504967 (+ 152us) write_op.cc:270] Start()
1211 20:55:16.504986 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514113358 usec, L: 0
1211 20:55:16.505683 (+ 697us) write_op.cc:270] Start()
1211 20:55:16.505705 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514135357 usec, L: 0
1211 20:55:16.505890 (+ 185us) write_op.cc:270] Start()
1211 20:55:16.505913 (+ 23us) write_op.cc:276] Timestamp: P: 1765486514147203 usec, L: 0
1211 20:55:16.506092 (+ 179us) write_op.cc:270] Start()
1211 20:55:16.506113 (+ 21us) write_op.cc:276] Timestamp: P: 1765486514159303 usec, L: 0
1211 20:55:16.506343 (+ 230us) write_op.cc:270] Start()
1211 20:55:16.506383 (+ 40us) write_op.cc:276] Timestamp: P: 1765486514189674 usec, L: 0
1211 20:55:16.506542 (+ 159us) write_op.cc:270] Start()
1211 20:55:16.506563 (+ 21us) write_op.cc:276] Timestamp: P: 1765486514243009 usec, L: 0
1211 20:55:16.506724 (+ 161us) write_op.cc:270] Start()
1211 20:55:16.506743 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514259451 usec, L: 0
1211 20:55:16.506896 (+ 153us) write_op.cc:270] Start()
1211 20:55:16.506917 (+ 21us) write_op.cc:276] Timestamp: P: 1765486514269105 usec, L: 0
1211 20:55:16.507100 (+ 183us) write_op.cc:270] Start()
1211 20:55:16.507121 (+ 21us) write_op.cc:276] Timestamp: P: 1765486514317577 usec, L: 0
1211 20:55:16.507290 (+ 169us) write_op.cc:270] Start()
1211 20:55:16.507312 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514330650 usec, L: 0
1211 20:55:16.507498 (+ 186us) write_op.cc:270] Start()
1211 20:55:16.507520 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514342379 usec, L: 0
1211 20:55:16.507695 (+ 175us) write_op.cc:270] Start()
1211 20:55:16.507718 (+ 23us) write_op.cc:276] Timestamp: P: 1765486514365854 usec, L: 0
1211 20:55:16.507883 (+ 165us) write_op.cc:270] Start()
1211 20:55:16.507908 (+ 25us) write_op.cc:276] Timestamp: P: 1765486514385120 usec, L: 0
1211 20:55:16.508086 (+ 178us) write_op.cc:270] Start()
1211 20:55:16.508108 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514389666 usec, L: 0
1211 20:55:16.508350 (+ 242us) write_op.cc:270] Start()
1211 20:55:16.508372 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514408830 usec, L: 0
1211 20:55:16.508552 (+ 180us) write_op.cc:270] Start()
1211 20:55:16.508573 (+ 21us) write_op.cc:276] Timestamp: P: 1765486514430287 usec, L: 0
1211 20:55:16.508711 (+ 138us) write_op.cc:270] Start()
1211 20:55:16.508733 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514436567 usec, L: 0
1211 20:55:16.508871 (+ 138us) write_op.cc:270] Start()
1211 20:55:16.508889 (+ 18us) write_op.cc:276] Timestamp: P: 1765486514442724 usec, L: 0
1211 20:55:16.509096 (+ 207us) write_op.cc:270] Start()
1211 20:55:16.509117 (+ 21us) write_op.cc:276] Timestamp: P: 1765486514463480 usec, L: 0
1211 20:55:16.509276 (+ 159us) write_op.cc:270] Start()
1211 20:55:16.509295 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514481953 usec, L: 0
1211 20:55:16.509429 (+ 134us) write_op.cc:270] Start()
1211 20:55:16.509446 (+ 17us) write_op.cc:276] Timestamp: P: 1765486514500377 usec, L: 0
1211 20:55:16.509588 (+ 142us) write_op.cc:270] Start()
1211 20:55:16.509609 (+ 21us) write_op.cc:276] Timestamp: P: 1765486514507860 usec, L: 0
1211 20:55:16.509737 (+ 128us) write_op.cc:270] Start()
1211 20:55:16.509756 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514519156 usec, L: 0
1211 20:55:16.509890 (+ 134us) write_op.cc:270] Start()
1211 20:55:16.509910 (+ 20us) write_op.cc:276] Timestamp: P: 1765486514547307 usec, L: 0
1211 20:55:16.547095 (+ 37185us) write_op.cc:270] Start()
1211 20:55:16.547120 (+ 25us) write_op.cc:276] Timestamp: P: 1765486514548575 usec, L: 0
1211 20:55:16.556562 (+ 9442us) write_op.cc:270] Start()
1211 20:55:16.556588 (+ 26us) write_op.cc:276] Timestamp: P: 1765486514568602 usec, L: 0
1211 20:55:16.560557 (+ 3969us) write_op.cc:270] Start()
1211 20:55:16.560582 (+ 25us) write_op.cc:276] Timestamp: P: 1765486514575957 usec, L: 0
1211 20:55:16.591883 (+ 31301us) write_op.cc:270] Start()
1211 20:55:16.591911 (+ 28us) write_op.cc:276] Timestamp: P: 1765486514589970 usec, L: 0
1211 20:55:16.592113 (+ 202us) write_op.cc:270] Start()
1211 20:55:16.592132 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514613117 usec, L: 0
1211 20:55:16.602076 (+ 9944us) write_op.cc:270] Start()
1211 20:55:16.602098 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514639929 usec, L: 0
1211 20:55:16.602245 (+ 147us) write_op.cc:270] Start()
1211 20:55:16.602262 (+ 17us) write_op.cc:276] Timestamp: P: 1765486514647296 usec, L: 0
1211 20:55:16.602404 (+ 142us) write_op.cc:270] Start()
1211 20:55:16.602423 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514662277 usec, L: 0
1211 20:55:16.602563 (+ 140us) write_op.cc:270] Start()
1211 20:55:16.602579 (+ 16us) write_op.cc:276] Timestamp: P: 1765486514697187 usec, L: 0
1211 20:55:16.602710 (+ 131us) write_op.cc:270] Start()
1211 20:55:16.602725 (+ 15us) write_op.cc:276] Timestamp: P: 1765486514708294 usec, L: 0
1211 20:55:16.602871 (+ 146us) write_op.cc:270] Start()
1211 20:55:16.602893 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514720820 usec, L: 0
1211 20:55:16.603385 (+ 492us) write_op.cc:270] Start()
1211 20:55:16.603418 (+ 33us) write_op.cc:276] Timestamp: P: 1765486514750613 usec, L: 0
1211 20:55:16.603593 (+ 175us) write_op.cc:270] Start()
1211 20:55:16.603612 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514776482 usec, L: 0
1211 20:55:16.603780 (+ 168us) write_op.cc:270] Start()
1211 20:55:16.603797 (+ 17us) write_op.cc:276] Timestamp: P: 1765486514780030 usec, L: 0
1211 20:55:16.603966 (+ 169us) write_op.cc:270] Start()
1211 20:55:16.603984 (+ 18us) write_op.cc:276] Timestamp: P: 1765486514783863 usec, L: 0
1211 20:55:16.604141 (+ 157us) write_op.cc:270] Start()
1211 20:55:16.604183 (+ 42us) write_op.cc:276] Timestamp: P: 1765486514794617 usec, L: 0
1211 20:55:16.631743 (+ 27560us) write_op.cc:270] Start()
1211 20:55:16.631769 (+ 26us) write_op.cc:276] Timestamp: P: 1765486514822549 usec, L: 0
1211 20:55:16.631966 (+ 197us) write_op.cc:270] Start()
1211 20:55:16.631988 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514896004 usec, L: 0
1211 20:55:16.636500 (+ 4512us) write_op.cc:270] Start()
1211 20:55:16.636524 (+ 24us) write_op.cc:276] Timestamp: P: 1765486514896591 usec, L: 0
1211 20:55:16.639594 (+ 3070us) write_op.cc:270] Start()
1211 20:55:16.639619 (+ 25us) write_op.cc:276] Timestamp: P: 1765486514897194 usec, L: 0
1211 20:55:16.641847 (+ 2228us) write_op.cc:270] Start()
1211 20:55:16.641867 (+ 20us) write_op.cc:276] Timestamp: P: 1765486514905635 usec, L: 0
1211 20:55:16.642833 (+ 966us) write_op.cc:270] Start()
1211 20:55:16.642859 (+ 26us) write_op.cc:276] Timestamp: P: 1765486514961285 usec, L: 0
1211 20:55:16.645228 (+ 2369us) write_op.cc:270] Start()
1211 20:55:16.645257 (+ 29us) write_op.cc:276] Timestamp: P: 1765486514963938 usec, L: 0
1211 20:55:16.659995 (+ 14738us) write_op.cc:270] Start()
1211 20:55:16.660024 (+ 29us) write_op.cc:276] Timestamp: P: 1765486514971273 usec, L: 0
1211 20:55:16.661494 (+ 1470us) write_op.cc:270] Start()
1211 20:55:16.661519 (+ 25us) write_op.cc:276] Timestamp: P: 1765486514998705 usec, L: 0
1211 20:55:16.664038 (+ 2519us) write_op.cc:270] Start()
1211 20:55:16.664064 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515012251 usec, L: 0
1211 20:55:16.693558 (+ 29494us) write_op.cc:270] Start()
1211 20:55:16.693587 (+ 29us) write_op.cc:276] Timestamp: P: 1765486515022670 usec, L: 0
1211 20:55:16.705206 (+ 11619us) write_op.cc:270] Start()
1211 20:55:16.705237 (+ 31us) write_op.cc:276] Timestamp: P: 1765486515046559 usec, L: 0
1211 20:55:16.705462 (+ 225us) write_op.cc:270] Start()
1211 20:55:16.705489 (+ 27us) write_op.cc:276] Timestamp: P: 1765486515053232 usec, L: 0
1211 20:55:16.709513 (+ 4024us) write_op.cc:270] Start()
1211 20:55:16.709534 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515104413 usec, L: 0
1211 20:55:16.717924 (+ 8390us) write_op.cc:270] Start()
1211 20:55:16.717955 (+ 31us) write_op.cc:276] Timestamp: P: 1765486515119505 usec, L: 0
1211 20:55:16.728967 (+ 11012us) write_op.cc:270] Start()
1211 20:55:16.728995 (+ 28us) write_op.cc:276] Timestamp: P: 1765486515153001 usec, L: 0
1211 20:55:16.743785 (+ 14790us) write_op.cc:270] Start()
1211 20:55:16.743823 (+ 38us) write_op.cc:276] Timestamp: P: 1765486515157016 usec, L: 0
1211 20:55:16.744691 (+ 868us) write_op.cc:270] Start()
1211 20:55:16.744717 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515198442 usec, L: 0
1211 20:55:16.744925 (+ 208us) write_op.cc:270] Start()
1211 20:55:16.744952 (+ 27us) write_op.cc:276] Timestamp: P: 1765486515206376 usec, L: 0
1211 20:55:16.745165 (+ 213us) write_op.cc:270] Start()
1211 20:55:16.745189 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515237125 usec, L: 0
1211 20:55:16.745397 (+ 208us) write_op.cc:270] Start()
1211 20:55:16.745421 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515238734 usec, L: 0
1211 20:55:16.749925 (+ 4504us) write_op.cc:270] Start()
1211 20:55:16.749953 (+ 28us) write_op.cc:276] Timestamp: P: 1765486515265520 usec, L: 0
1211 20:55:16.750233 (+ 280us) write_op.cc:270] Start()
1211 20:55:16.750257 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515267803 usec, L: 0
1211 20:55:16.750528 (+ 271us) write_op.cc:270] Start()
1211 20:55:16.750565 (+ 37us) write_op.cc:276] Timestamp: P: 1765486515313187 usec, L: 0
1211 20:55:16.750782 (+ 217us) write_op.cc:270] Start()
1211 20:55:16.750807 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515328721 usec, L: 0
1211 20:55:16.751034 (+ 227us) write_op.cc:270] Start()
1211 20:55:16.751058 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515331677 usec, L: 0
1211 20:55:16.751282 (+ 224us) write_op.cc:270] Start()
1211 20:55:16.751305 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515343494 usec, L: 0
1211 20:55:16.751512 (+ 207us) write_op.cc:270] Start()
1211 20:55:16.751535 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515380059 usec, L: 0
1211 20:55:16.751730 (+ 195us) write_op.cc:270] Start()
1211 20:55:16.751752 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515382620 usec, L: 0
1211 20:55:16.751951 (+ 199us) write_op.cc:270] Start()
1211 20:55:16.751974 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515386004 usec, L: 0
1211 20:55:16.752213 (+ 239us) write_op.cc:270] Start()
1211 20:55:16.752236 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515404176 usec, L: 0
1211 20:55:16.752439 (+ 203us) write_op.cc:270] Start()
1211 20:55:16.752462 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515440168 usec, L: 0
1211 20:55:16.752672 (+ 210us) write_op.cc:270] Start()
1211 20:55:16.752698 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515453778 usec, L: 0
1211 20:55:16.752919 (+ 221us) write_op.cc:270] Start()
1211 20:55:16.752942 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515464599 usec, L: 0
1211 20:55:16.753211 (+ 269us) write_op.cc:270] Start()
1211 20:55:16.753236 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515481399 usec, L: 0
1211 20:55:16.753484 (+ 248us) write_op.cc:270] Start()
1211 20:55:16.753510 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515493090 usec, L: 0
1211 20:55:16.753727 (+ 217us) write_op.cc:270] Start()
1211 20:55:16.753771 (+ 44us) write_op.cc:276] Timestamp: P: 1765486515507304 usec, L: 0
1211 20:55:16.769367 (+ 15596us) write_op.cc:270] Start()
1211 20:55:16.769395 (+ 28us) write_op.cc:276] Timestamp: P: 1765486515519221 usec, L: 0
1211 20:55:16.774606 (+ 5211us) write_op.cc:270] Start()
1211 20:55:16.774633 (+ 27us) write_op.cc:276] Timestamp: P: 1765486515548192 usec, L: 0
1211 20:55:16.828985 (+ 54352us) write_op.cc:270] Start()
1211 20:55:16.829004 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515557938 usec, L: 0
1211 20:55:16.829379 (+ 375us) write_op.cc:270] Start()
1211 20:55:16.829404 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515569629 usec, L: 0
1211 20:55:16.829620 (+ 216us) write_op.cc:270] Start()
1211 20:55:16.829640 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515587999 usec, L: 0
1211 20:55:16.829852 (+ 212us) write_op.cc:270] Start()
1211 20:55:16.829878 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515599716 usec, L: 0
1211 20:55:16.830082 (+ 204us) write_op.cc:270] Start()
1211 20:55:16.830103 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515637788 usec, L: 0
1211 20:55:16.830279 (+ 176us) write_op.cc:270] Start()
1211 20:55:16.830304 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515644023 usec, L: 0
1211 20:55:16.832535 (+ 2231us) write_op.cc:270] Start()
1211 20:55:16.832561 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515646530 usec, L: 0
1211 20:55:16.857956 (+ 25395us) write_op.cc:270] Start()
1211 20:55:16.857985 (+ 29us) write_op.cc:276] Timestamp: P: 1765486515649481 usec, L: 0
1211 20:55:16.858227 (+ 242us) write_op.cc:270] Start()
1211 20:55:16.858251 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515672426 usec, L: 0
1211 20:55:16.932867 (+ 74616us) write_op.cc:270] Start()
1211 20:55:16.932897 (+ 30us) write_op.cc:276] Timestamp: P: 1765486515698927 usec, L: 0
1211 20:55:16.936841 (+ 3944us) write_op.cc:270] Start()
1211 20:55:16.936863 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515701482 usec, L: 0
1211 20:55:16.939739 (+ 2876us) write_op.cc:270] Start()
1211 20:55:16.939760 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515728216 usec, L: 0
1211 20:55:16.939933 (+ 173us) write_op.cc:270] Start()
1211 20:55:16.939955 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515750967 usec, L: 0
1211 20:55:16.957250 (+ 17295us) write_op.cc:270] Start()
1211 20:55:16.957273 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515751834 usec, L: 0
1211 20:55:16.963636 (+ 6363us) write_op.cc:270] Start()
1211 20:55:16.963662 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515771792 usec, L: 0
1211 20:55:16.970307 (+ 6645us) write_op.cc:270] Start()
1211 20:55:16.970329 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515780594 usec, L: 0
1211 20:55:16.970524 (+ 195us) write_op.cc:270] Start()
1211 20:55:16.970545 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515797866 usec, L: 0
1211 20:55:16.974467 (+ 3922us) write_op.cc:270] Start()
1211 20:55:16.974490 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515840515 usec, L: 0
1211 20:55:16.981431 (+ 6941us) write_op.cc:270] Start()
1211 20:55:16.981463 (+ 32us) write_op.cc:276] Timestamp: P: 1765486515864435 usec, L: 0
1211 20:55:17.024633 (+ 43170us) write_op.cc:270] Start()
1211 20:55:17.024657 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515865532 usec, L: 0
1211 20:55:17.032012 (+ 7355us) write_op.cc:270] Start()
1211 20:55:17.032054 (+ 42us) write_op.cc:276] Timestamp: P: 1765486515866427 usec, L: 0
1211 20:55:17.033637 (+ 1583us) write_op.cc:270] Start()
1211 20:55:17.033666 (+ 29us) write_op.cc:276] Timestamp: P: 1765486515877242 usec, L: 0
1211 20:55:17.036386 (+ 2720us) write_op.cc:270] Start()
1211 20:55:17.036413 (+ 27us) write_op.cc:276] Timestamp: P: 1765486515924245 usec, L: 0
1211 20:55:17.059053 (+ 22640us) write_op.cc:270] Start()
1211 20:55:17.059080 (+ 27us) write_op.cc:276] Timestamp: P: 1765486515935305 usec, L: 0
1211 20:55:17.061065 (+ 1985us) write_op.cc:270] Start()
1211 20:55:17.061086 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515955466 usec, L: 0
1211 20:55:17.066049 (+ 4963us) write_op.cc:270] Start()
1211 20:55:17.066091 (+ 42us) write_op.cc:276] Timestamp: P: 1765486515985257 usec, L: 0
1211 20:55:17.076089 (+ 9998us) write_op.cc:270] Start()
1211 20:55:17.076115 (+ 26us) write_op.cc:276] Timestamp: P: 1765486516005298 usec, L: 0
1211 20:55:17.080874 (+ 4759us) write_op.cc:270] Start()
1211 20:55:17.080906 (+ 32us) write_op.cc:276] Timestamp: P: 1765486516013293 usec, L: 0
1211 20:55:17.083716 (+ 2810us) write_op.cc:270] Start()
1211 20:55:17.083739 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516036131 usec, L: 0
1211 20:55:17.094739 (+ 11000us) write_op.cc:270] Start()
1211 20:55:17.094764 (+ 25us) write_op.cc:276] Timestamp: P: 1765486516060575 usec, L: 0
1211 20:55:17.094911 (+ 147us) write_op.cc:270] Start()
1211 20:55:17.094948 (+ 37us) write_op.cc:276] Timestamp: P: 1765486516080048 usec, L: 0
1211 20:55:17.095113 (+ 165us) write_op.cc:270] Start()
1211 20:55:17.095177 (+ 64us) write_op.cc:276] Timestamp: P: 1765486516101646 usec, L: 0
1211 20:55:17.095634 (+ 457us) write_op.cc:270] Start()
1211 20:55:17.095660 (+ 26us) write_op.cc:276] Timestamp: P: 1765486516103455 usec, L: 0
1211 20:55:17.095856 (+ 196us) write_op.cc:270] Start()
1211 20:55:17.095880 (+ 24us) write_op.cc:276] Timestamp: P: 1765486516125780 usec, L: 0
1211 20:55:17.096044 (+ 164us) write_op.cc:270] Start()
1211 20:55:17.096080 (+ 36us) write_op.cc:276] Timestamp: P: 1765486516127515 usec, L: 0
1211 20:55:17.096234 (+ 154us) write_op.cc:270] Start()
1211 20:55:17.096254 (+ 20us) write_op.cc:276] Timestamp: P: 1765486516168490 usec, L: 0
1211 20:55:17.096414 (+ 160us) write_op.cc:270] Start()
1211 20:55:17.096433 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516171276 usec, L: 0
1211 20:55:17.096604 (+ 171us) write_op.cc:270] Start()
1211 20:55:17.096624 (+ 20us) write_op.cc:276] Timestamp: P: 1765486516179668 usec, L: 0
1211 20:55:17.096769 (+ 145us) write_op.cc:270] Start()
1211 20:55:17.096787 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516191372 usec, L: 0
1211 20:55:17.096936 (+ 149us) write_op.cc:270] Start()
1211 20:55:17.096956 (+ 20us) write_op.cc:276] Timestamp: P: 1765486516224371 usec, L: 0
1211 20:55:17.097092 (+ 136us) write_op.cc:270] Start()
1211 20:55:17.097109 (+ 17us) write_op.cc:276] Timestamp: P: 1765486516249658 usec, L: 0
1211 20:55:17.097271 (+ 162us) write_op.cc:270] Start()
1211 20:55:17.097293 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516262935 usec, L: 0
1211 20:55:17.097465 (+ 172us) write_op.cc:270] Start()
1211 20:55:17.097483 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516263673 usec, L: 0
1211 20:55:17.097611 (+ 128us) write_op.cc:270] Start()
1211 20:55:17.097628 (+ 17us) write_op.cc:276] Timestamp: P: 1765486516281586 usec, L: 0
1211 20:55:17.152868 (+ 55240us) log.cc:844] Serialized 97208 byte log entry
1211 20:55:17.196473 (+ 43605us) raft_consensus.cc:1645] Marking committed up to 1320
1211 20:55:17.357330 (+160857us) raft_consensus.cc:1697] Filling consensus response to leader.
1211 20:55:17.358785 (+ 1455us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
1211 20:55:17.358798 (+ 13us) raft_consensus.cc:1685] finished
1211 20:55:17.358800 (+ 2us) raft_consensus.cc:1691] UpdateReplicas() finished
1211 20:55:17.358884 (+ 84us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:55:17.196520 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196545 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196565 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196585 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196604 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196623 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196642 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196661 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196681 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196699 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196716 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196734 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196752 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196768 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196785 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196801 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196818 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196836 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196857 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196877 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196895 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196913 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196930 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196953 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196972 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.196995 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.197015 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.197033 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.197050 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.197066 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.197082 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.208975 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209033 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209064 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209094 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209122 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209147 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209176 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209205 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209233 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209264 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209293 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209333 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209361 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209401 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209430 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209458 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209486 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209531 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209561 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209589 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209618 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209646 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209675 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209703 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209733 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209761 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209791 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209821 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209850 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209880 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209907 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209936 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209964 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.209994 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210053 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210083 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210112 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210139 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210168 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210199 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210229 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210259 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210289 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210319 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210346 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210383 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210412 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210441 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210469 (+ 0us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:17.210499 (+ 0
I20251211 20:55:17.965296 7051 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:17.980314 7051 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:17.995765 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1253 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:18.003547 7051 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:18.006026 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1253 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:18.014055 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:18.014499 7051 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:17.500622 6946 rpcz_store.cc:276] 1211 20:55:15.080140 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:15.080188 (+ 48us) service_pool.cc:225] Handling call
1211 20:55:15.261055 (+180867us) raft_consensus.cc:1471] Updating replica for 369 ops
1211 20:55:15.454467 (+193412us) raft_consensus.cc:1534] Early marking committed up to index 884
1211 20:55:15.455031 (+ 564us) raft_consensus.cc:1539] Triggering prepare for 369 ops
1211 20:55:15.466413 (+ 11382us) write_op.cc:270] Start()
1211 20:55:15.466440 (+ 27us) write_op.cc:276] Timestamp: P: 1765486508213265 usec, L: 0
1211 20:55:15.466608 (+ 168us) write_op.cc:270] Start()
1211 20:55:15.466627 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508220195 usec, L: 0
1211 20:55:15.466766 (+ 139us) write_op.cc:270] Start()
1211 20:55:15.466784 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508260720 usec, L: 0
1211 20:55:15.467067 (+ 283us) write_op.cc:270] Start()
1211 20:55:15.467089 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508265360 usec, L: 0
1211 20:55:15.475181 (+ 8092us) write_op.cc:270] Start()
1211 20:55:15.475217 (+ 36us) write_op.cc:276] Timestamp: P: 1765486508268211 usec, L: 0
1211 20:55:15.493660 (+ 18443us) write_op.cc:270] Start()
1211 20:55:15.493686 (+ 26us) write_op.cc:276] Timestamp: P: 1765486508279671 usec, L: 0
1211 20:55:15.493857 (+ 171us) write_op.cc:270] Start()
1211 20:55:15.497124 (+ 3267us) write_op.cc:276] Timestamp: P: 1765486508313186 usec, L: 0
1211 20:55:15.528311 (+ 31187us) write_op.cc:270] Start()
1211 20:55:15.528334 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508322467 usec, L: 0
1211 20:55:15.528551 (+ 217us) write_op.cc:270] Start()
1211 20:55:15.528570 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508326709 usec, L: 0
1211 20:55:15.528712 (+ 142us) write_op.cc:270] Start()
1211 20:55:15.528730 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508342352 usec, L: 0
1211 20:55:15.528868 (+ 138us) write_op.cc:270] Start()
1211 20:55:15.528886 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508355707 usec, L: 0
1211 20:55:15.529021 (+ 135us) write_op.cc:270] Start()
1211 20:55:15.529039 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508392002 usec, L: 0
1211 20:55:15.529177 (+ 138us) write_op.cc:270] Start()
1211 20:55:15.529194 (+ 17us) write_op.cc:276] Timestamp: P: 1765486508401551 usec, L: 0
1211 20:55:15.529329 (+ 135us) write_op.cc:270] Start()
1211 20:55:15.529348 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508415784 usec, L: 0
1211 20:55:15.529479 (+ 131us) write_op.cc:270] Start()
1211 20:55:15.529497 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508428968 usec, L: 0
1211 20:55:15.529627 (+ 130us) write_op.cc:270] Start()
1211 20:55:15.529647 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508457608 usec, L: 0
1211 20:55:15.529794 (+ 147us) write_op.cc:270] Start()
1211 20:55:15.529812 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508470740 usec, L: 0
1211 20:55:15.529964 (+ 152us) write_op.cc:270] Start()
1211 20:55:15.529983 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508476125 usec, L: 0
1211 20:55:15.530287 (+ 304us) write_op.cc:270] Start()
1211 20:55:15.530306 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508496419 usec, L: 0
1211 20:55:15.530441 (+ 135us) write_op.cc:270] Start()
1211 20:55:15.530460 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508504405 usec, L: 0
1211 20:55:15.530601 (+ 141us) write_op.cc:270] Start()
1211 20:55:15.530620 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508520021 usec, L: 0
1211 20:55:15.530767 (+ 147us) write_op.cc:270] Start()
1211 20:55:15.530788 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508528125 usec, L: 0
1211 20:55:15.530961 (+ 173us) write_op.cc:270] Start()
1211 20:55:15.530981 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508562529 usec, L: 0
1211 20:55:15.531152 (+ 171us) write_op.cc:270] Start()
1211 20:55:15.531172 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508565952 usec, L: 0
1211 20:55:15.539940 (+ 8768us) write_op.cc:270] Start()
1211 20:55:15.539962 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508581135 usec, L: 0
1211 20:55:15.575781 (+ 35819us) write_op.cc:270] Start()
1211 20:55:15.575804 (+ 23us) write_op.cc:276] Timestamp: P: 1765486508607205 usec, L: 0
1211 20:55:15.579019 (+ 3215us) write_op.cc:270] Start()
1211 20:55:15.579035 (+ 16us) write_op.cc:276] Timestamp: P: 1765486508615897 usec, L: 0
1211 20:55:15.596064 (+ 17029us) write_op.cc:270] Start()
1211 20:55:15.596086 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508640240 usec, L: 0
1211 20:55:15.598618 (+ 2532us) write_op.cc:270] Start()
1211 20:55:15.598638 (+ 20us) write_op.cc:276] Timestamp: P: 1765486508652616 usec, L: 0
1211 20:55:15.598796 (+ 158us) write_op.cc:270] Start()
1211 20:55:15.598815 (+ 19us) write_op.cc:276] Timestamp: P: 1765486508653809 usec, L: 0
1211 20:55:15.598974 (+ 159us) write_op.cc:270] Start()
1211 20:55:15.598995 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508710324 usec, L: 0
1211 20:55:15.625550 (+ 26555us) write_op.cc:270] Start()
1211 20:55:15.625568 (+ 18us) write_op.cc:276] Timestamp: P: 1765486508724219 usec, L: 0
1211 20:55:15.629242 (+ 3674us) write_op.cc:270] Start()
1211 20:55:15.629269 (+ 27us) write_op.cc:276] Timestamp: P: 1765486508728674 usec, L: 0
1211 20:55:15.629432 (+ 163us) write_op.cc:270] Start()
1211 20:55:15.629458 (+ 26us) write_op.cc:276] Timestamp: P: 1765486508781942 usec, L: 0
1211 20:55:15.657297 (+ 27839us) write_op.cc:270] Start()
1211 20:55:15.657323 (+ 26us) write_op.cc:276] Timestamp: P: 1765486508814709 usec, L: 0
1211 20:55:15.665061 (+ 7738us) write_op.cc:270] Start()
1211 20:55:15.665087 (+ 26us) write_op.cc:276] Timestamp: P: 1765486508817211 usec, L: 0
1211 20:55:15.665250 (+ 163us) write_op.cc:270] Start()
1211 20:55:15.665271 (+ 21us) write_op.cc:276] Timestamp: P: 1765486508817811 usec, L: 0
1211 20:55:15.668463 (+ 3192us) write_op.cc:270] Start()
1211 20:55:15.668489 (+ 26us) write_op.cc:276] Timestamp: P: 1765486508876680 usec, L: 0
1211 20:55:15.688603 (+ 20114us) write_op.cc:270] Start()
1211 20:55:15.688625 (+ 22us) write_op.cc:276] Timestamp: P: 1765486508900340 usec, L: 0
1211 20:55:15.690700 (+ 2075us) write_op.cc:270] Start()
1211 20:55:15.690724 (+ 24us) write_op.cc:276] Timestamp: P: 1765486508904603 usec, L: 0
1211 20:55:15.690911 (+ 187us) write_op.cc:270] Start()
1211 20:55:15.690936 (+ 25us) write_op.cc:276] Timestamp: P: 1765486508912246 usec, L: 0
1211 20:55:15.691045 (+ 109us) write_op.cc:270] Start()
1211 20:55:15.691060 (+ 15us) write_op.cc:276] Timestamp: P: 1765486508967073 usec, L: 0
1211 20:55:15.709644 (+ 18584us) write_op.cc:270] Start()
1211 20:55:15.709666 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509000896 usec, L: 0
1211 20:55:15.709841 (+ 175us) write_op.cc:270] Start()
1211 20:55:15.709861 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509003868 usec, L: 0
1211 20:55:15.710015 (+ 154us) write_op.cc:270] Start()
1211 20:55:15.710037 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509035681 usec, L: 0
1211 20:55:15.710193 (+ 156us) write_op.cc:270] Start()
1211 20:55:15.710214 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509054836 usec, L: 0
1211 20:55:15.710662 (+ 448us) write_op.cc:270] Start()
1211 20:55:15.710688 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509089428 usec, L: 0
1211 20:55:15.710857 (+ 169us) write_op.cc:270] Start()
1211 20:55:15.710877 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509099648 usec, L: 0
1211 20:55:15.711179 (+ 302us) write_op.cc:270] Start()
1211 20:55:15.711201 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509140742 usec, L: 0
1211 20:55:15.711358 (+ 157us) write_op.cc:270] Start()
1211 20:55:15.711379 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509164705 usec, L: 0
1211 20:55:15.711509 (+ 130us) write_op.cc:270] Start()
1211 20:55:15.711527 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509165240 usec, L: 0
1211 20:55:15.731569 (+ 20042us) write_op.cc:270] Start()
1211 20:55:15.731596 (+ 27us) write_op.cc:276] Timestamp: P: 1765486509196542 usec, L: 0
1211 20:55:15.735156 (+ 3560us) write_op.cc:270] Start()
1211 20:55:15.735181 (+ 25us) write_op.cc:276] Timestamp: P: 1765486509220087 usec, L: 0
1211 20:55:15.740945 (+ 5764us) write_op.cc:270] Start()
1211 20:55:15.740969 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509241873 usec, L: 0
1211 20:55:15.743177 (+ 2208us) write_op.cc:270] Start()
1211 20:55:15.743206 (+ 29us) write_op.cc:276] Timestamp: P: 1765486509255767 usec, L: 0
1211 20:55:15.743404 (+ 198us) write_op.cc:270] Start()
1211 20:55:15.743428 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509259442 usec, L: 0
1211 20:55:15.743621 (+ 193us) write_op.cc:270] Start()
1211 20:55:15.743645 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509297753 usec, L: 0
1211 20:55:15.743816 (+ 171us) write_op.cc:270] Start()
1211 20:55:15.743842 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509298392 usec, L: 0
1211 20:55:15.743995 (+ 153us) write_op.cc:270] Start()
1211 20:55:15.744013 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509330556 usec, L: 0
1211 20:55:15.744152 (+ 139us) write_op.cc:270] Start()
1211 20:55:15.744216 (+ 64us) write_op.cc:276] Timestamp: P: 1765486509338813 usec, L: 0
1211 20:55:15.744400 (+ 184us) write_op.cc:270] Start()
1211 20:55:15.744422 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509348030 usec, L: 0
1211 20:55:15.744572 (+ 150us) write_op.cc:270] Start()
1211 20:55:15.744593 (+ 21us) write_op.cc:276] Timestamp: P: 1765486509373549 usec, L: 0
1211 20:55:15.744747 (+ 154us) write_op.cc:270] Start()
1211 20:55:15.744769 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509390443 usec, L: 0
1211 20:55:15.744942 (+ 173us) write_op.cc:270] Start()
1211 20:55:15.744962 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509412385 usec, L: 0
1211 20:55:15.745090 (+ 128us) write_op.cc:270] Start()
1211 20:55:15.745107 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509432728 usec, L: 0
1211 20:55:15.745203 (+ 96us) write_op.cc:270] Start()
1211 20:55:15.745216 (+ 13us) write_op.cc:276] Timestamp: P: 1765486509436134 usec, L: 0
1211 20:55:15.745311 (+ 95us) write_op.cc:270] Start()
1211 20:55:15.745324 (+ 13us) write_op.cc:276] Timestamp: P: 1765486509441814 usec, L: 0
1211 20:55:15.745447 (+ 123us) write_op.cc:270] Start()
1211 20:55:15.745460 (+ 13us) write_op.cc:276] Timestamp: P: 1765486509451256 usec, L: 0
1211 20:55:15.780832 (+ 35372us) write_op.cc:270] Start()
1211 20:55:15.780855 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509470610 usec, L: 0
1211 20:55:15.790506 (+ 9651us) write_op.cc:270] Start()
1211 20:55:15.790528 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509483488 usec, L: 0
1211 20:55:15.799687 (+ 9159us) write_op.cc:270] Start()
1211 20:55:15.799710 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509504797 usec, L: 0
1211 20:55:15.799847 (+ 137us) write_op.cc:270] Start()
1211 20:55:15.799866 (+ 19us) write_op.cc:276] Timestamp: P: 1765486509522439 usec, L: 0
1211 20:55:15.800009 (+ 143us) write_op.cc:270] Start()
1211 20:55:15.800026 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509527719 usec, L: 0
1211 20:55:15.800176 (+ 150us) write_op.cc:270] Start()
1211 20:55:15.800196 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509546045 usec, L: 0
1211 20:55:15.819442 (+ 19246us) write_op.cc:270] Start()
1211 20:55:15.819464 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509567307 usec, L: 0
1211 20:55:15.820976 (+ 1512us) write_op.cc:270] Start()
1211 20:55:15.821002 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509584586 usec, L: 0
1211 20:55:15.825679 (+ 4677us) write_op.cc:270] Start()
1211 20:55:15.825708 (+ 29us) write_op.cc:276] Timestamp: P: 1765486509585233 usec, L: 0
1211 20:55:15.839550 (+ 13842us) write_op.cc:270] Start()
1211 20:55:15.839575 (+ 25us) write_op.cc:276] Timestamp: P: 1765486509604493 usec, L: 0
1211 20:55:15.839728 (+ 153us) write_op.cc:270] Start()
1211 20:55:15.839750 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509617761 usec, L: 0
1211 20:55:15.839901 (+ 151us) write_op.cc:270] Start()
1211 20:55:15.839919 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509660095 usec, L: 0
1211 20:55:15.840206 (+ 287us) write_op.cc:270] Start()
1211 20:55:15.840226 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509663759 usec, L: 0
1211 20:55:15.840384 (+ 158us) write_op.cc:270] Start()
1211 20:55:15.840402 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509679933 usec, L: 0
1211 20:55:15.840558 (+ 156us) write_op.cc:270] Start()
1211 20:55:15.840575 (+ 17us) write_op.cc:276] Timestamp: P: 1765486509683841 usec, L: 0
1211 20:55:15.840705 (+ 130us) write_op.cc:270] Start()
1211 20:55:15.840723 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509730889 usec, L: 0
1211 20:55:15.840856 (+ 133us) write_op.cc:270] Start()
1211 20:55:15.840879 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509733449 usec, L: 0
1211 20:55:15.841025 (+ 146us) write_op.cc:270] Start()
1211 20:55:15.841047 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509781096 usec, L: 0
1211 20:55:15.841201 (+ 154us) write_op.cc:270] Start()
1211 20:55:15.841221 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509801418 usec, L: 0
1211 20:55:15.841348 (+ 127us) write_op.cc:270] Start()
1211 20:55:15.841368 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509814824 usec, L: 0
1211 20:55:15.841521 (+ 153us) write_op.cc:270] Start()
1211 20:55:15.841541 (+ 20us) write_op.cc:276] Timestamp: P: 1765486509822684 usec, L: 0
1211 20:55:15.846592 (+ 5051us) write_op.cc:270] Start()
1211 20:55:15.846616 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509862675 usec, L: 0
1211 20:55:15.862835 (+ 16219us) write_op.cc:270] Start()
1211 20:55:15.862858 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509872391 usec, L: 0
1211 20:55:15.870184 (+ 7326us) write_op.cc:270] Start()
1211 20:55:15.870208 (+ 24us) write_op.cc:276] Timestamp: P: 1765486509893827 usec, L: 0
1211 20:55:15.870374 (+ 166us) write_op.cc:270] Start()
1211 20:55:15.870392 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509897426 usec, L: 0
1211 20:55:15.870521 (+ 129us) write_op.cc:270] Start()
1211 20:55:15.870554 (+ 33us) write_op.cc:276] Timestamp: P: 1765486509921643 usec, L: 0
1211 20:55:15.870705 (+ 151us) write_op.cc:270] Start()
1211 20:55:15.870721 (+ 16us) write_op.cc:276] Timestamp: P: 1765486509931785 usec, L: 0
1211 20:55:15.870854 (+ 133us) write_op.cc:270] Start()
1211 20:55:15.870872 (+ 18us) write_op.cc:276] Timestamp: P: 1765486509981154 usec, L: 0
1211 20:55:15.873655 (+ 2783us) write_op.cc:270] Start()
1211 20:55:15.873681 (+ 26us) write_op.cc:276] Timestamp: P: 1765486509981883 usec, L: 0
1211 20:55:15.877538 (+ 3857us) write_op.cc:270] Start()
1211 20:55:15.877560 (+ 22us) write_op.cc:276] Timestamp: P: 1765486509986128 usec, L: 0
1211 20:55:15.877713 (+ 153us) write_op.cc:270] Start()
1211 20:55:15.877736 (+ 23us) write_op.cc:276] Timestamp: P: 1765486509986965 usec, L: 0
1211 20:55:15.877882 (+ 146us) write_op.cc:270] Start()
1211 20:55:15.877903 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510030736 usec, L: 0
1211 20:55:15.878071 (+ 168us) write_op.cc:270] Start()
1211 20:55:15.878091 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510031653 usec, L: 0
1211 20:55:15.878239 (+ 148us) write_op.cc:270] Start()
1211 20:55:15.878257 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510067743 usec, L: 0
1211 20:55:15.879928 (+ 1671us) write_op.cc:270] Start()
1211 20:55:15.879949 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510099655 usec, L: 0
1211 20:55:15.880080 (+ 131us) write_op.cc:270] Start()
1211 20:55:15.880106 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510113595 usec, L: 0
1211 20:55:15.880219 (+ 113us) write_op.cc:270] Start()
1211 20:55:15.880238 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510117846 usec, L: 0
1211 20:55:15.880345 (+ 107us) write_op.cc:270] Start()
1211 20:55:15.880368 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510160280 usec, L: 0
1211 20:55:15.880498 (+ 130us) write_op.cc:270] Start()
1211 20:55:15.880520 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510208351 usec, L: 0
1211 20:55:15.880629 (+ 109us) write_op.cc:270] Start()
1211 20:55:15.880643 (+ 14us) write_op.cc:276] Timestamp: P: 1765486510216291 usec, L: 0
1211 20:55:15.880737 (+ 94us) write_op.cc:270] Start()
1211 20:55:15.880751 (+ 14us) write_op.cc:276] Timestamp: P: 1765486510227720 usec, L: 0
1211 20:55:15.880852 (+ 101us) write_op.cc:270] Start()
1211 20:55:15.880865 (+ 13us) write_op.cc:276] Timestamp: P: 1765486510241625 usec, L: 0
1211 20:55:15.880975 (+ 110us) write_op.cc:270] Start()
1211 20:55:15.880991 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510275402 usec, L: 0
1211 20:55:15.881120 (+ 129us) write_op.cc:270] Start()
1211 20:55:15.881136 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510310322 usec, L: 0
1211 20:55:15.881241 (+ 105us) write_op.cc:270] Start()
1211 20:55:15.881255 (+ 14us) write_op.cc:276] Timestamp: P: 1765486510340082 usec, L: 0
1211 20:55:15.881361 (+ 106us) write_op.cc:270] Start()
1211 20:55:15.881375 (+ 14us) write_op.cc:276] Timestamp: P: 1765486510363126 usec, L: 0
1211 20:55:15.881472 (+ 97us) write_op.cc:270] Start()
1211 20:55:15.881485 (+ 13us) write_op.cc:276] Timestamp: P: 1765486510363865 usec, L: 0
1211 20:55:15.881606 (+ 121us) write_op.cc:270] Start()
1211 20:55:15.881622 (+ 16us) write_op.cc:276] Timestamp: P: 1765486510389812 usec, L: 0
1211 20:55:15.881723 (+ 101us) write_op.cc:270] Start()
1211 20:55:15.881738 (+ 15us) write_op.cc:276] Timestamp: P: 1765486510433563 usec, L: 0
1211 20:55:15.881853 (+ 115us) write_op.cc:270] Start()
1211 20:55:15.881868 (+ 15us) write_op.cc:276] Timestamp: P: 1765486510442130 usec, L: 0
1211 20:55:15.881974 (+ 106us) write_op.cc:270] Start()
1211 20:55:15.882011 (+ 37us) write_op.cc:276] Timestamp: P: 1765486510483452 usec, L: 0
1211 20:55:15.882155 (+ 144us) write_op.cc:270] Start()
1211 20:55:15.882176 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510484062 usec, L: 0
1211 20:55:15.882316 (+ 140us) write_op.cc:270] Start()
1211 20:55:15.882334 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510487471 usec, L: 0
1211 20:55:15.882470 (+ 136us) write_op.cc:270] Start()
1211 20:55:15.882497 (+ 27us) write_op.cc:276] Timestamp: P: 1765486510517998 usec, L: 0
1211 20:55:15.882656 (+ 159us) write_op.cc:270] Start()
1211 20:55:15.882679 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510554426 usec, L: 0
1211 20:55:15.882827 (+ 148us) write_op.cc:270] Start()
1211 20:55:15.882844 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510562378 usec, L: 0
1211 20:55:15.883000 (+ 156us) write_op.cc:270] Start()
1211 20:55:15.883019 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510570826 usec, L: 0
1211 20:55:15.883170 (+ 151us) write_op.cc:270] Start()
1211 20:55:15.883189 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510610919 usec, L: 0
1211 20:55:15.924761 (+ 41572us) write_op.cc:270] Start()
1211 20:55:15.924786 (+ 25us) write_op.cc:276] Timestamp: P: 1765486510611630 usec, L: 0
1211 20:55:15.942718 (+ 17932us) write_op.cc:270] Start()
1211 20:55:15.942745 (+ 27us) write_op.cc:276] Timestamp: P: 1765486510647904 usec, L: 0
1211 20:55:15.966987 (+ 24242us) write_op.cc:270] Start()
1211 20:55:15.967009 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510665889 usec, L: 0
1211 20:55:15.972807 (+ 5798us) write_op.cc:270] Start()
1211 20:55:15.972833 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510690743 usec, L: 0
1211 20:55:15.973017 (+ 184us) write_op.cc:270] Start()
1211 20:55:15.975271 (+ 2254us) write_op.cc:276] Timestamp: P: 1765486510699303 usec, L: 0
1211 20:55:15.980481 (+ 5210us) write_op.cc:270] Start()
1211 20:55:15.980508 (+ 27us) write_op.cc:276] Timestamp: P: 1765486510721802 usec, L: 0
1211 20:55:15.984086 (+ 3578us) write_op.cc:270] Start()
1211 20:55:15.984144 (+ 58us) write_op.cc:276] Timestamp: P: 1765486510746479 usec, L: 0
1211 20:55:15.990728 (+ 6584us) write_op.cc:270] Start()
1211 20:55:15.990752 (+ 24us) write_op.cc:276] Timestamp: P: 1765486510747116 usec, L: 0
1211 20:55:15.991865 (+ 1113us) write_op.cc:270] Start()
1211 20:55:15.991888 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510748986 usec, L: 0
1211 20:55:15.992036 (+ 148us) write_op.cc:270] Start()
1211 20:55:15.992056 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510776193 usec, L: 0
1211 20:55:15.993172 (+ 1116us) write_op.cc:270] Start()
1211 20:55:15.993198 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510805610 usec, L: 0
1211 20:55:15.998704 (+ 5506us) write_op.cc:270] Start()
1211 20:55:15.998752 (+ 48us) write_op.cc:276] Timestamp: P: 1765486510813416 usec, L: 0
1211 20:55:15.998904 (+ 152us) write_op.cc:270] Start()
1211 20:55:15.998923 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510814433 usec, L: 0
1211 20:55:15.999073 (+ 150us) write_op.cc:270] Start()
1211 20:55:15.999092 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510819659 usec, L: 0
1211 20:55:15.999239 (+ 147us) write_op.cc:270] Start()
1211 20:55:15.999258 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510840747 usec, L: 0
1211 20:55:15.999397 (+ 139us) write_op.cc:270] Start()
1211 20:55:15.999414 (+ 17us) write_op.cc:276] Timestamp: P: 1765486510889207 usec, L: 0
1211 20:55:15.999564 (+ 150us) write_op.cc:270] Start()
1211 20:55:15.999583 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510966874 usec, L: 0
1211 20:55:16.000434 (+ 851us) write_op.cc:270] Start()
1211 20:55:16.000455 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510982149 usec, L: 0
1211 20:55:16.000592 (+ 137us) write_op.cc:270] Start()
1211 20:55:16.000611 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511005349 usec, L: 0
1211 20:55:16.000764 (+ 153us) write_op.cc:270] Start()
1211 20:55:16.000782 (+ 18us) write_op.cc:276] Timestamp: P: 1765486511007340 usec, L: 0
1211 20:55:16.000890 (+ 108us) write_op.cc:270] Start()
1211 20:55:16.000906 (+ 16us) write_op.cc:276] Timestamp: P: 1765486511039113 usec, L: 0
1211 20:55:16.001062 (+ 156us) write_op.cc:270] Start()
1211 20:55:16.001079 (+ 17us) write_op.cc:276] Timestamp: P: 1765486511075474 usec, L: 0
1211 20:55:16.001229 (+ 150us) write_op.cc:270] Start()
1211 20:55:16.001245 (+ 16us) write_op.cc:276] Timestamp: P: 1765486511094244 usec, L: 0
1211 20:55:16.001406 (+ 161us) write_op.cc:270] Start()
1211 20:55:16.001424 (+ 18us) write_op.cc:276] Timestamp: P: 1765486511113524 usec, L: 0
1211 20:55:16.001547 (+ 123us) write_op.cc:270] Start()
1211 20:55:16.001564 (+ 17us) write_op.cc:276] Timestamp: P: 1765486511118944 usec, L: 0
1211 20:55:16.001672 (+ 108us) write_op.cc:270] Start()
1211 20:55:16.001686 (+ 14us) write_op.cc:276] Timestamp: P: 1765486511119655 usec, L: 0
1211 20:55:16.001815 (+ 129us) write_op.cc:270] Start()
1211 20:55:16.001832 (+ 17us) write_op.cc:276] Timestamp: P: 1765486511153624 usec, L: 0
1211 20:55:16.011980 (+ 10148us) write_op.cc:270] Start()
1211 20:55:16.012022 (+ 42us) write_op.cc:276] Timestamp: P: 1765486511198124 usec, L: 0
1211 20:55:16.012221 (+ 199us) write_op.cc:270] Start()
1211 20:55:16.012242 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511216843 usec, L: 0
1211 20:55:16.049379 (+ 37137us) write_op.cc:270] Start()
1211 20:55:16.049403 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511227191 usec, L: 0
1211 20:55:16.049569 (+ 166us) write_op.cc:270] Start()
1211 20:55:16.049589 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511231013 usec, L: 0
1211 20:55:16.049754 (+ 165us) write_op.cc:270] Start()
1211 20:55:16.049775 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511273486 usec, L: 0
1211 20:55:16.049932 (+ 157us) write_op.cc:270] Start()
1211 20:55:16.049950 (+ 18us) write_op.cc:276] Timestamp: P: 1765486511286455 usec, L: 0
1211 20:55:16.050627 (+ 677us) write_op.cc:270] Start()
1211 20:55:16.050649 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511314161 usec, L: 0
1211 20:55:16.058558 (+ 7909us) write_op.cc:270] Start()
1211 20:55:16.058579 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511316702 usec, L: 0
1211 20:55:16.066622 (+ 8043us) write_op.cc:270] Start()
1211 20:55:16.066645 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511355555 usec, L: 0
1211 20:55:16.073600 (+ 6955us) write_op.cc:270] Start()
1211 20:55:16.073624 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511363796 usec, L: 0
1211 20:55:16.076702 (+ 3078us) write_op.cc:270] Start()
1211 20:55:16.076728 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511396130 usec, L: 0
1211 20:55:16.078655 (+ 1927us) write_op.cc:270] Start()
1211 20:55:16.078680 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511398962 usec, L: 0
1211 20:55:16.088351 (+ 9671us) write_op.cc:270] Start()
1211 20:55:16.088379 (+ 28us) write_op.cc:276] Timestamp: P: 1765486511400506 usec, L: 0
1211 20:55:16.111533 (+ 23154us) write_op.cc:270] Start()
1211 20:55:16.111576 (+ 43us) write_op.cc:276] Timestamp: P: 1765486511465149 usec, L: 0
1211 20:55:16.115248 (+ 3672us) write_op.cc:270] Start()
1211 20:55:16.115275 (+ 27us) write_op.cc:276] Timestamp: P: 1765486511482736 usec, L: 0
1211 20:55:16.115458 (+ 183us) write_op.cc:270] Start()
1211 20:55:16.115484 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511490911 usec, L: 0
1211 20:55:16.115661 (+ 177us) write_op.cc:270] Start()
1211 20:55:16.115703 (+ 42us) write_op.cc:276] Timestamp: P: 1765486511500469 usec, L: 0
1211 20:55:16.117879 (+ 2176us) write_op.cc:270] Start()
1211 20:55:16.117902 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511521215 usec, L: 0
1211 20:55:16.118113 (+ 211us) write_op.cc:270] Start()
1211 20:55:16.118135 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511548148 usec, L: 0
1211 20:55:16.118340 (+ 205us) write_op.cc:270] Start()
1211 20:55:16.118370 (+ 30us) write_op.cc:276] Timestamp: P: 1765486511575655 usec, L: 0
1211 20:55:16.120068 (+ 1698us) write_op.cc:270] Start()
1211 20:55:16.120097 (+ 29us) write_op.cc:276] Timestamp: P: 1765486511580366 usec, L: 0
1211 20:55:16.120738 (+ 641us) write_op.cc:270] Start()
1211 20:55:16.120770 (+ 32us) write_op.cc:276] Timestamp: P: 1765486511581546 usec, L: 0
1211 20:55:16.120988 (+ 218us) write_op.cc:270] Start()
1211 20:55:16.121012 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511621046 usec, L: 0
1211 20:55:16.121223 (+ 211us) write_op.cc:270] Start()
1211 20:55:16.121246 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511662788 usec, L: 0
1211 20:55:16.121445 (+ 199us) write_op.cc:270] Start()
1211 20:55:16.121469 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511670500 usec, L: 0
1211 20:55:16.121639 (+ 170us) write_op.cc:270] Start()
1211 20:55:16.121662 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511672436 usec, L: 0
1211 20:55:16.144093 (+ 22431us) write_op.cc:270] Start()
1211 20:55:16.144119 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511673264 usec, L: 0
1211 20:55:16.144880 (+ 761us) write_op.cc:270] Start()
1211 20:55:16.144907 (+ 27us) write_op.cc:276] Timestamp: P: 1765486511714468 usec, L: 0
1211 20:55:16.145095 (+ 188us) write_op.cc:270] Start()
1211 20:55:16.145128 (+ 33us) write_op.cc:276] Timestamp: P: 1765486511739224 usec, L: 0
1211 20:55:16.150570 (+ 5442us) write_op.cc:270] Start()
1211 20:55:16.150594 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511760787 usec, L: 0
1211 20:55:16.150780 (+ 186us) write_op.cc:270] Start()
1211 20:55:16.150801 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511765758 usec, L: 0
1211 20:55:16.159286 (+ 8485us) write_op.cc:270] Start()
1211 20:55:16.159311 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511770092 usec, L: 0
1211 20:55:16.159678 (+ 367us) write_op.cc:270] Start()
1211 20:55:16.166495 (+ 6817us) write_op.cc:276] Timestamp: P: 1765486511803075 usec, L: 0
1211 20:55:16.172303 (+ 5808us) write_op.cc:270] Start()
1211 20:55:16.172327 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511815270 usec, L: 0
1211 20:55:16.177572 (+ 5245us) write_op.cc:270] Start()
1211 20:55:16.177595 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511833302 usec, L: 0
1211 20:55:16.191318 (+ 13723us) write_op.cc:270] Start()
1211 20:55:16.191345 (+ 27us) write_op.cc:276] Timestamp: P: 1765486511846378 usec, L: 0
1211 20:55:16.191538 (+ 193us) write_op.cc:270] Start()
1211 20:55:16.191562 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511859443 usec, L: 0
1211 20:55:16.191740 (+ 178us) write_op.cc:270] Start()
1211 20:55:16.191764 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511891104 usec, L: 0
1211 20:55:16.191912 (+ 148us) write_op.cc:270] Start()
1211 20:55:16.191937 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511898126 usec, L: 0
1211 20:55:16.192130 (+ 193us) write_op.cc:270] Start()
1211 20:55:16.192153 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511926284 usec, L: 0
1211 20:55:16.194882 (+ 2729us) write_op.cc:270] Start()
1211 20:55:16.194913 (+ 31us) write_op.cc:276] Timestamp: P: 1765486511960082 usec, L: 0
1211 20:55:16.195121 (+ 208us) write_op.cc:270] Start()
1211 20:55:16.195146 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511976618 usec, L: 0
1211 20:55:16.195864 (+ 718us) write_op.cc:270] Start()
1211 20:55:16.195889 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511988582 usec, L: 0
1211 20:55:16.196159 (+ 270us) write_op.cc:270] Start()
1211 20:55:16.196186 (+ 27us) write_op.cc:276] Timestamp: P: 176548
I20251211 20:55:18.445429 7073 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:18.465232 7073 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:18.475672 7073 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:18.475670 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1253 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:18.476243 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1253 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:18.481766 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:18.498590 7073 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:18.672235 6940 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.194:47609 (request call id 21) took 1138 ms. Trace:
I20251211 20:55:18.685667 7051 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:18.685848 7051 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:18.687510 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1376 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:18.688879 7051 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:18.692090 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1376 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:18.694818 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:18.695052 7051 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:18.672374 6940 rpcz_store.cc:276] 1211 20:55:17.529921 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:17.530193 (+ 272us) service_pool.cc:225] Handling call
1211 20:55:17.663153 (+132960us) raft_consensus.cc:1471] Updating replica for 172 ops
1211 20:55:17.709250 (+ 46097us) raft_consensus.cc:1534] Early marking committed up to index 1204
1211 20:55:17.709514 (+ 264us) raft_consensus.cc:1539] Triggering prepare for 172 ops
1211 20:55:17.719150 (+ 9636us) write_op.cc:270] Start()
1211 20:55:17.719174 (+ 24us) write_op.cc:276] Timestamp: P: 1765486514225390 usec, L: 0
1211 20:55:17.719336 (+ 162us) write_op.cc:270] Start()
1211 20:55:17.719355 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514253099 usec, L: 0
1211 20:55:17.719499 (+ 144us) write_op.cc:270] Start()
1211 20:55:17.719517 (+ 18us) write_op.cc:276] Timestamp: P: 1765486514258864 usec, L: 0
1211 20:55:17.719630 (+ 113us) write_op.cc:270] Start()
1211 20:55:17.719647 (+ 17us) write_op.cc:276] Timestamp: P: 1765486514263321 usec, L: 0
1211 20:55:17.719780 (+ 133us) write_op.cc:270] Start()
1211 20:55:17.719800 (+ 20us) write_op.cc:276] Timestamp: P: 1765486514321514 usec, L: 0
1211 20:55:17.719953 (+ 153us) write_op.cc:270] Start()
1211 20:55:17.719973 (+ 20us) write_op.cc:276] Timestamp: P: 1765486514342980 usec, L: 0
1211 20:55:17.720129 (+ 156us) write_op.cc:270] Start()
1211 20:55:17.720164 (+ 35us) write_op.cc:276] Timestamp: P: 1765486514359012 usec, L: 0
1211 20:55:17.728785 (+ 8621us) write_op.cc:270] Start()
1211 20:55:17.728812 (+ 27us) write_op.cc:276] Timestamp: P: 1765486514367503 usec, L: 0
1211 20:55:17.728945 (+ 133us) write_op.cc:270] Start()
1211 20:55:17.728959 (+ 14us) write_op.cc:276] Timestamp: P: 1765486514379827 usec, L: 0
1211 20:55:17.739196 (+ 10237us) write_op.cc:270] Start()
1211 20:55:17.739215 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514388716 usec, L: 0
1211 20:55:17.747548 (+ 8333us) write_op.cc:270] Start()
1211 20:55:17.747572 (+ 24us) write_op.cc:276] Timestamp: P: 1765486514423598 usec, L: 0
1211 20:55:17.756504 (+ 8932us) write_op.cc:270] Start()
1211 20:55:17.756525 (+ 21us) write_op.cc:276] Timestamp: P: 1765486514428845 usec, L: 0
1211 20:55:17.761064 (+ 4539us) write_op.cc:270] Start()
1211 20:55:17.761083 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514438259 usec, L: 0
1211 20:55:17.762965 (+ 1882us) write_op.cc:270] Start()
1211 20:55:17.762987 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514441052 usec, L: 0
1211 20:55:17.771732 (+ 8745us) write_op.cc:270] Start()
1211 20:55:17.774200 (+ 2468us) write_op.cc:276] Timestamp: P: 1765486514461314 usec, L: 0
1211 20:55:17.777918 (+ 3718us) write_op.cc:270] Start()
1211 20:55:17.777942 (+ 24us) write_op.cc:276] Timestamp: P: 1765486514484881 usec, L: 0
1211 20:55:17.779401 (+ 1459us) write_op.cc:270] Start()
1211 20:55:17.779425 (+ 24us) write_op.cc:276] Timestamp: P: 1765486514493494 usec, L: 0
1211 20:55:17.790719 (+ 11294us) write_op.cc:270] Start()
1211 20:55:17.790744 (+ 25us) write_op.cc:276] Timestamp: P: 1765486514505368 usec, L: 0
1211 20:55:17.791169 (+ 425us) write_op.cc:270] Start()
1211 20:55:17.791199 (+ 30us) write_op.cc:276] Timestamp: P: 1765486514527611 usec, L: 0
1211 20:55:17.791377 (+ 178us) write_op.cc:270] Start()
1211 20:55:17.791397 (+ 20us) write_op.cc:276] Timestamp: P: 1765486514547968 usec, L: 0
1211 20:55:17.791537 (+ 140us) write_op.cc:270] Start()
1211 20:55:17.791557 (+ 20us) write_op.cc:276] Timestamp: P: 1765486514548807 usec, L: 0
1211 20:55:17.791682 (+ 125us) write_op.cc:270] Start()
1211 20:55:17.791701 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514557737 usec, L: 0
1211 20:55:17.791863 (+ 162us) write_op.cc:270] Start()
1211 20:55:17.791890 (+ 27us) write_op.cc:276] Timestamp: P: 1765486514577283 usec, L: 0
1211 20:55:17.792024 (+ 134us) write_op.cc:270] Start()
1211 20:55:17.792041 (+ 17us) write_op.cc:276] Timestamp: P: 1765486514599339 usec, L: 0
1211 20:55:17.792185 (+ 144us) write_op.cc:270] Start()
1211 20:55:17.792204 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514612890 usec, L: 0
1211 20:55:17.818437 (+ 26233us) write_op.cc:270] Start()
1211 20:55:17.818461 (+ 24us) write_op.cc:276] Timestamp: P: 1765486514626458 usec, L: 0
1211 20:55:17.818651 (+ 190us) write_op.cc:270] Start()
1211 20:55:17.818673 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514636387 usec, L: 0
1211 20:55:17.818833 (+ 160us) write_op.cc:270] Start()
1211 20:55:17.818851 (+ 18us) write_op.cc:276] Timestamp: P: 1765486514659398 usec, L: 0
1211 20:55:17.818988 (+ 137us) write_op.cc:270] Start()
1211 20:55:17.819005 (+ 17us) write_op.cc:276] Timestamp: P: 1765486514688341 usec, L: 0
1211 20:55:17.819139 (+ 134us) write_op.cc:270] Start()
1211 20:55:17.819156 (+ 17us) write_op.cc:276] Timestamp: P: 1765486514712912 usec, L: 0
1211 20:55:17.819312 (+ 156us) write_op.cc:270] Start()
1211 20:55:17.819330 (+ 18us) write_op.cc:276] Timestamp: P: 1765486514730992 usec, L: 0
1211 20:55:17.819500 (+ 170us) write_op.cc:270] Start()
1211 20:55:17.819535 (+ 35us) write_op.cc:276] Timestamp: P: 1765486514747202 usec, L: 0
1211 20:55:17.819724 (+ 189us) write_op.cc:270] Start()
1211 20:55:17.820593 (+ 869us) write_op.cc:276] Timestamp: P: 1765486514764149 usec, L: 0
1211 20:55:17.820766 (+ 173us) write_op.cc:270] Start()
1211 20:55:17.820786 (+ 20us) write_op.cc:276] Timestamp: P: 1765486514784447 usec, L: 0
1211 20:55:17.820967 (+ 181us) write_op.cc:270] Start()
1211 20:55:17.820985 (+ 18us) write_op.cc:276] Timestamp: P: 1765486514785167 usec, L: 0
1211 20:55:17.821157 (+ 172us) write_op.cc:270] Start()
1211 20:55:17.821175 (+ 18us) write_op.cc:276] Timestamp: P: 1765486514794522 usec, L: 0
1211 20:55:17.821339 (+ 164us) write_op.cc:270] Start()
1211 20:55:17.821360 (+ 21us) write_op.cc:276] Timestamp: P: 1765486514833252 usec, L: 0
1211 20:55:17.821486 (+ 126us) write_op.cc:270] Start()
1211 20:55:17.821503 (+ 17us) write_op.cc:276] Timestamp: P: 1765486514895751 usec, L: 0
1211 20:55:17.821652 (+ 149us) write_op.cc:270] Start()
1211 20:55:17.821671 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514896433 usec, L: 0
1211 20:55:17.821826 (+ 155us) write_op.cc:270] Start()
1211 20:55:17.821846 (+ 20us) write_op.cc:276] Timestamp: P: 1765486514903987 usec, L: 0
1211 20:55:17.821979 (+ 133us) write_op.cc:270] Start()
1211 20:55:17.822018 (+ 39us) write_op.cc:276] Timestamp: P: 1765486514905912 usec, L: 0
1211 20:55:17.822206 (+ 188us) write_op.cc:270] Start()
1211 20:55:17.822225 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514960655 usec, L: 0
1211 20:55:17.822368 (+ 143us) write_op.cc:270] Start()
1211 20:55:17.822390 (+ 22us) write_op.cc:276] Timestamp: P: 1765486514963983 usec, L: 0
1211 20:55:17.822526 (+ 136us) write_op.cc:270] Start()
1211 20:55:17.822545 (+ 19us) write_op.cc:276] Timestamp: P: 1765486514971797 usec, L: 0
1211 20:55:17.822676 (+ 131us) write_op.cc:270] Start()
1211 20:55:17.822694 (+ 18us) write_op.cc:276] Timestamp: P: 1765486514993275 usec, L: 0
1211 20:55:17.822825 (+ 131us) write_op.cc:270] Start()
1211 20:55:17.822847 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515013339 usec, L: 0
1211 20:55:17.823008 (+ 161us) write_op.cc:270] Start()
1211 20:55:17.823026 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515022354 usec, L: 0
1211 20:55:17.823218 (+ 192us) write_op.cc:270] Start()
1211 20:55:17.823238 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515047216 usec, L: 0
1211 20:55:17.823360 (+ 122us) write_op.cc:270] Start()
1211 20:55:17.823377 (+ 17us) write_op.cc:276] Timestamp: P: 1765486515051946 usec, L: 0
1211 20:55:17.823524 (+ 147us) write_op.cc:270] Start()
1211 20:55:17.823541 (+ 17us) write_op.cc:276] Timestamp: P: 1765486515108339 usec, L: 0
1211 20:55:17.823712 (+ 171us) write_op.cc:270] Start()
1211 20:55:17.823737 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515120027 usec, L: 0
1211 20:55:17.823882 (+ 145us) write_op.cc:270] Start()
1211 20:55:17.823915 (+ 33us) write_op.cc:276] Timestamp: P: 1765486515153659 usec, L: 0
1211 20:55:17.824064 (+ 149us) write_op.cc:270] Start()
1211 20:55:17.824082 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515159266 usec, L: 0
1211 20:55:17.824220 (+ 138us) write_op.cc:270] Start()
1211 20:55:17.824238 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515197034 usec, L: 0
1211 20:55:17.824451 (+ 213us) write_op.cc:270] Start()
1211 20:55:17.824472 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515199940 usec, L: 0
1211 20:55:17.824603 (+ 131us) write_op.cc:270] Start()
1211 20:55:17.824621 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515237203 usec, L: 0
1211 20:55:17.824758 (+ 137us) write_op.cc:270] Start()
1211 20:55:17.824775 (+ 17us) write_op.cc:276] Timestamp: P: 1765486515263743 usec, L: 0
1211 20:55:17.824890 (+ 115us) write_op.cc:270] Start()
1211 20:55:17.824905 (+ 15us) write_op.cc:276] Timestamp: P: 1765486515266090 usec, L: 0
1211 20:55:17.825103 (+ 198us) write_op.cc:270] Start()
1211 20:55:17.825129 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515274871 usec, L: 0
1211 20:55:17.876231 (+ 51102us) write_op.cc:270] Start()
1211 20:55:17.880205 (+ 3974us) write_op.cc:276] Timestamp: P: 1765486515314223 usec, L: 0
1211 20:55:17.884094 (+ 3889us) write_op.cc:270] Start()
1211 20:55:17.884131 (+ 37us) write_op.cc:276] Timestamp: P: 1765486515329266 usec, L: 0
1211 20:55:17.885825 (+ 1694us) write_op.cc:270] Start()
1211 20:55:17.885865 (+ 40us) write_op.cc:276] Timestamp: P: 1765486515332269 usec, L: 0
1211 20:55:17.894258 (+ 8393us) write_op.cc:270] Start()
1211 20:55:17.894282 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515345137 usec, L: 0
1211 20:55:17.894490 (+ 208us) write_op.cc:270] Start()
1211 20:55:17.894511 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515380737 usec, L: 0
1211 20:55:17.911379 (+ 16868us) write_op.cc:270] Start()
1211 20:55:17.911404 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515387302 usec, L: 0
1211 20:55:17.911558 (+ 154us) write_op.cc:270] Start()
1211 20:55:17.911579 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515404978 usec, L: 0
1211 20:55:17.915982 (+ 4403us) write_op.cc:270] Start()
1211 20:55:17.916009 (+ 27us) write_op.cc:276] Timestamp: P: 1765486515430987 usec, L: 0
1211 20:55:17.916193 (+ 184us) write_op.cc:270] Start()
1211 20:55:17.916216 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515441430 usec, L: 0
1211 20:55:17.916382 (+ 166us) write_op.cc:270] Start()
1211 20:55:17.916403 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515460790 usec, L: 0
1211 20:55:17.916559 (+ 156us) write_op.cc:270] Start()
1211 20:55:17.916579 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515467352 usec, L: 0
1211 20:55:17.916757 (+ 178us) write_op.cc:270] Start()
1211 20:55:17.916778 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515477590 usec, L: 0
1211 20:55:17.916931 (+ 153us) write_op.cc:270] Start()
1211 20:55:17.916951 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515493425 usec, L: 0
1211 20:55:17.950186 (+ 33235us) write_op.cc:270] Start()
1211 20:55:17.950206 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515502928 usec, L: 0
1211 20:55:17.950332 (+ 126us) write_op.cc:270] Start()
1211 20:55:17.950348 (+ 16us) write_op.cc:276] Timestamp: P: 1765486515527285 usec, L: 0
1211 20:55:17.971119 (+ 20771us) write_op.cc:270] Start()
1211 20:55:17.971144 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515547558 usec, L: 0
1211 20:55:17.971332 (+ 188us) write_op.cc:270] Start()
1211 20:55:17.971378 (+ 46us) write_op.cc:276] Timestamp: P: 1765486515558499 usec, L: 0
1211 20:55:18.007765 (+ 36387us) write_op.cc:270] Start()
1211 20:55:18.007795 (+ 30us) write_op.cc:276] Timestamp: P: 1765486515572141 usec, L: 0
1211 20:55:18.025623 (+ 17828us) write_op.cc:270] Start()
1211 20:55:18.025653 (+ 30us) write_op.cc:276] Timestamp: P: 1765486515574427 usec, L: 0
1211 20:55:18.034769 (+ 9116us) write_op.cc:270] Start()
1211 20:55:18.034794 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515592973 usec, L: 0
1211 20:55:18.036318 (+ 1524us) write_op.cc:270] Start()
1211 20:55:18.036346 (+ 28us) write_op.cc:276] Timestamp: P: 1765486515634577 usec, L: 0
1211 20:55:18.047719 (+ 11373us) write_op.cc:270] Start()
1211 20:55:18.047744 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515641175 usec, L: 0
1211 20:55:18.078579 (+ 30835us) write_op.cc:270] Start()
1211 20:55:18.078606 (+ 27us) write_op.cc:276] Timestamp: P: 1765486515644603 usec, L: 0
1211 20:55:18.078777 (+ 171us) write_op.cc:270] Start()
1211 20:55:18.078799 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515645078 usec, L: 0
1211 20:55:18.078983 (+ 184us) write_op.cc:270] Start()
1211 20:55:18.079005 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515674301 usec, L: 0
1211 20:55:18.079203 (+ 198us) write_op.cc:270] Start()
1211 20:55:18.079224 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515696160 usec, L: 0
1211 20:55:18.079407 (+ 183us) write_op.cc:270] Start()
1211 20:55:18.079426 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515704974 usec, L: 0
1211 20:55:18.079605 (+ 179us) write_op.cc:270] Start()
1211 20:55:18.079630 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515731916 usec, L: 0
1211 20:55:18.088271 (+ 8641us) write_op.cc:270] Start()
1211 20:55:18.088298 (+ 27us) write_op.cc:276] Timestamp: P: 1765486515747384 usec, L: 0
1211 20:55:18.092503 (+ 4205us) write_op.cc:270] Start()
1211 20:55:18.092528 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515751341 usec, L: 0
1211 20:55:18.092688 (+ 160us) write_op.cc:270] Start()
1211 20:55:18.092708 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515773633 usec, L: 0
1211 20:55:18.113494 (+ 20786us) write_op.cc:270] Start()
1211 20:55:18.113518 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515784331 usec, L: 0
1211 20:55:18.143227 (+ 29709us) write_op.cc:270] Start()
1211 20:55:18.143252 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515801429 usec, L: 0
1211 20:55:18.143448 (+ 196us) write_op.cc:270] Start()
1211 20:55:18.143470 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515830127 usec, L: 0
1211 20:55:18.143619 (+ 149us) write_op.cc:270] Start()
1211 20:55:18.143638 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515854228 usec, L: 0
1211 20:55:18.143775 (+ 137us) write_op.cc:270] Start()
1211 20:55:18.143793 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515866194 usec, L: 0
1211 20:55:18.143983 (+ 190us) write_op.cc:270] Start()
1211 20:55:18.144003 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515874149 usec, L: 0
1211 20:55:18.144153 (+ 150us) write_op.cc:270] Start()
1211 20:55:18.144173 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515879924 usec, L: 0
1211 20:55:18.144329 (+ 156us) write_op.cc:270] Start()
1211 20:55:18.144347 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515926645 usec, L: 0
1211 20:55:18.146285 (+ 1938us) write_op.cc:270] Start()
1211 20:55:18.146329 (+ 44us) write_op.cc:276] Timestamp: P: 1765486515933803 usec, L: 0
1211 20:55:18.160741 (+ 14412us) write_op.cc:270] Start()
1211 20:55:18.160760 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515965435 usec, L: 0
1211 20:55:18.160883 (+ 123us) write_op.cc:270] Start()
1211 20:55:18.160898 (+ 15us) write_op.cc:276] Timestamp: P: 1765486515975702 usec, L: 0
1211 20:55:18.171060 (+ 10162us) write_op.cc:270] Start()
1211 20:55:18.171083 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515997794 usec, L: 0
1211 20:55:18.171226 (+ 143us) write_op.cc:270] Start()
1211 20:55:18.171244 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516008951 usec, L: 0
1211 20:55:18.171386 (+ 142us) write_op.cc:270] Start()
1211 20:55:18.171405 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516035461 usec, L: 0
1211 20:55:18.171536 (+ 131us) write_op.cc:270] Start()
1211 20:55:18.171550 (+ 14us) write_op.cc:276] Timestamp: P: 1765486516061330 usec, L: 0
1211 20:55:18.171682 (+ 132us) write_op.cc:270] Start()
1211 20:55:18.171697 (+ 15us) write_op.cc:276] Timestamp: P: 1765486516081293 usec, L: 0
1211 20:55:18.171820 (+ 123us) write_op.cc:270] Start()
1211 20:55:18.171839 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516096160 usec, L: 0
1211 20:55:18.171987 (+ 148us) write_op.cc:270] Start()
1211 20:55:18.172005 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516105041 usec, L: 0
1211 20:55:18.172155 (+ 150us) write_op.cc:270] Start()
1211 20:55:18.172176 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516114994 usec, L: 0
1211 20:55:18.172334 (+ 158us) write_op.cc:270] Start()
1211 20:55:18.172352 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516125191 usec, L: 0
1211 20:55:18.172481 (+ 129us) write_op.cc:270] Start()
1211 20:55:18.172496 (+ 15us) write_op.cc:276] Timestamp: P: 1765486516155478 usec, L: 0
1211 20:55:18.172593 (+ 97us) write_op.cc:270] Start()
1211 20:55:18.172607 (+ 14us) write_op.cc:276] Timestamp: P: 1765486516181019 usec, L: 0
1211 20:55:18.189334 (+ 16727us) write_op.cc:270] Start()
1211 20:55:18.189358 (+ 24us) write_op.cc:276] Timestamp: P: 1765486516182027 usec, L: 0
1211 20:55:18.216926 (+ 27568us) write_op.cc:270] Start()
1211 20:55:18.216953 (+ 27us) write_op.cc:276] Timestamp: P: 1765486516190493 usec, L: 0
1211 20:55:18.217116 (+ 163us) write_op.cc:270] Start()
1211 20:55:18.217137 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516225025 usec, L: 0
1211 20:55:18.217265 (+ 128us) write_op.cc:270] Start()
1211 20:55:18.217281 (+ 16us) write_op.cc:276] Timestamp: P: 1765486516246218 usec, L: 0
1211 20:55:18.217449 (+ 168us) write_op.cc:270] Start()
1211 20:55:18.217466 (+ 17us) write_op.cc:276] Timestamp: P: 1765486516262320 usec, L: 0
1211 20:55:18.217617 (+ 151us) write_op.cc:270] Start()
1211 20:55:18.217636 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516263614 usec, L: 0
1211 20:55:18.217796 (+ 160us) write_op.cc:270] Start()
1211 20:55:18.217815 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516282498 usec, L: 0
1211 20:55:18.217954 (+ 139us) write_op.cc:270] Start()
1211 20:55:18.217971 (+ 17us) write_op.cc:276] Timestamp: P: 1765486516336614 usec, L: 0
1211 20:55:18.218154 (+ 183us) write_op.cc:270] Start()
1211 20:55:18.218173 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516348720 usec, L: 0
1211 20:55:18.218304 (+ 131us) write_op.cc:270] Start()
1211 20:55:18.218321 (+ 17us) write_op.cc:276] Timestamp: P: 1765486516354495 usec, L: 0
1211 20:55:18.218480 (+ 159us) write_op.cc:270] Start()
1211 20:55:18.218499 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516383476 usec, L: 0
1211 20:55:18.218637 (+ 138us) write_op.cc:270] Start()
1211 20:55:18.218656 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516402572 usec, L: 0
1211 20:55:18.218798 (+ 142us) write_op.cc:270] Start()
1211 20:55:18.218817 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516422762 usec, L: 0
1211 20:55:18.218953 (+ 136us) write_op.cc:270] Start()
1211 20:55:18.218973 (+ 20us) write_op.cc:276] Timestamp: P: 1765486516435720 usec, L: 0
1211 20:55:18.219249 (+ 276us) write_op.cc:270] Start()
1211 20:55:18.219269 (+ 20us) write_op.cc:276] Timestamp: P: 1765486516457397 usec, L: 0
1211 20:55:18.221631 (+ 2362us) write_op.cc:270] Start()
1211 20:55:18.221672 (+ 41us) write_op.cc:276] Timestamp: P: 1765486516481112 usec, L: 0
1211 20:55:18.229354 (+ 7682us) write_op.cc:270] Start()
1211 20:55:18.229390 (+ 36us) write_op.cc:276] Timestamp: P: 1765486516510937 usec, L: 0
1211 20:55:18.229563 (+ 173us) write_op.cc:270] Start()
1211 20:55:18.229586 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516531023 usec, L: 0
1211 20:55:18.234298 (+ 4712us) write_op.cc:270] Start()
1211 20:55:18.241692 (+ 7394us) write_op.cc:276] Timestamp: P: 1765486516559118 usec, L: 0
1211 20:55:18.242900 (+ 1208us) write_op.cc:270] Start()
1211 20:55:18.242923 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516579637 usec, L: 0
1211 20:55:18.243918 (+ 995us) write_op.cc:270] Start()
1211 20:55:18.243944 (+ 26us) write_op.cc:276] Timestamp: P: 1765486516587170 usec, L: 0
1211 20:55:18.275938 (+ 31994us) write_op.cc:270] Start()
1211 20:55:18.275962 (+ 24us) write_op.cc:276] Timestamp: P: 1765486516590390 usec, L: 0
1211 20:55:18.276152 (+ 190us) write_op.cc:270] Start()
1211 20:55:18.276174 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516636691 usec, L: 0
1211 20:55:18.276327 (+ 153us) write_op.cc:270] Start()
1211 20:55:18.276346 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516669832 usec, L: 0
1211 20:55:18.276497 (+ 151us) write_op.cc:270] Start()
1211 20:55:18.276515 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516670403 usec, L: 0
1211 20:55:18.276643 (+ 128us) write_op.cc:270] Start()
1211 20:55:18.276661 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516680759 usec, L: 0
1211 20:55:18.276812 (+ 151us) write_op.cc:270] Start()
1211 20:55:18.276833 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516689203 usec, L: 0
1211 20:55:18.276982 (+ 149us) write_op.cc:270] Start()
1211 20:55:18.277000 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516713114 usec, L: 0
1211 20:55:18.277151 (+ 151us) write_op.cc:270] Start()
1211 20:55:18.277170 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516729848 usec, L: 0
1211 20:55:18.277314 (+ 144us) write_op.cc:270] Start()
1211 20:55:18.277331 (+ 17us) write_op.cc:276] Timestamp: P: 1765486516746228 usec, L: 0
1211 20:55:18.277477 (+ 146us) write_op.cc:270] Start()
1211 20:55:18.277495 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516762563 usec, L: 0
1211 20:55:18.277621 (+ 126us) write_op.cc:270] Start()
1211 20:55:18.277642 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516776893 usec, L: 0
1211 20:55:18.277777 (+ 135us) write_op.cc:270] Start()
1211 20:55:18.277794 (+ 17us) write_op.cc:276] Timestamp: P: 1765486516798349 usec, L: 0
1211 20:55:18.294261 (+ 16467us) write_op.cc:270] Start()
1211 20:55:18.294286 (+ 25us) write_op.cc:276] Timestamp: P: 1765486516817556 usec, L: 0
1211 20:55:18.294474 (+ 188us) write_op.cc:270] Start()
1211 20:55:18.294490 (+ 16us) write_op.cc:276] Timestamp: P: 1765486516819624 usec, L: 0
1211 20:55:18.294635 (+ 145us) write_op.cc:270] Start()
1211 20:55:18.294657 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516860931 usec, L: 0
1211 20:55:18.294854 (+ 197us) write_op.cc:270] Start()
1211 20:55:18.294874 (+ 20us) write_op.cc:276] Timestamp: P: 1765486516888909 usec, L: 0
1211 20:55:18.294997 (+ 123us) write_op.cc:270] Start()
1211 20:55:18.295013 (+ 16us) write_op.cc:276] Timestamp: P: 1765486516910000 usec, L: 0
1211 20:55:18.295113 (+ 100us) write_op.cc:270] Start()
1211 20:55:18.295139 (+ 26us) write_op.cc:276] Timestamp: P: 1765486516911215 usec, L: 0
1211 20:55:18.295279 (+ 140us) write_op.cc:270] Start()
1211 20:55:18.295297 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516922547 usec, L: 0
1211 20:55:18.296568 (+ 1271us) write_op.cc:270] Start()
1211 20:55:18.296592 (+ 24us) write_op.cc:276] Timestamp: P: 1765486516964970 usec, L: 0
1211 20:55:18.296772 (+ 180us) write_op.cc:270] Start()
1211 20:55:18.296793 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516986012 usec, L: 0
1211 20:55:18.322911 (+ 26118us) write_op.cc:270] Start()
1211 20:55:18.322936 (+ 25us) write_op.cc:276] Timestamp: P: 1765486516987497 usec, L: 0
1211 20:55:18.328319 (+ 5383us) write_op.cc:270] Start()
1211 20:55:18.328343 (+ 24us) write_op.cc:276] Timestamp: P: 1765486516988565 usec, L: 0
1211 20:55:18.331327 (+ 2984us) write_op.cc:270] Start()
1211 20:55:18.331348 (+ 21us) write_op.cc:276] Timestamp: P: 1765486517010187 usec, L: 0
1211 20:55:18.336382 (+ 5034us) write_op.cc:270] Start()
1211 20:55:18.336405 (+ 23us) write_op.cc:276] Timestamp: P: 1765486517040856 usec, L: 0
1211 20:55:18.362706 (+ 26301us) write_op.cc:270] Start()
1211 20:55:18.362736 (+ 30us) write_op.cc:276] Timestamp: P: 1765486517052602 usec, L: 0
1211 20:55:18.362886 (+ 150us) write_op.cc:270] Start()
1211 20:55:18.362905 (+ 19us) write_op.cc:276] Timestamp: P: 1765486517077820 usec, L: 0
1211 20:55:18.363067 (+ 162us) write_op.cc:270] Start()
1211 20:55:18.363086 (+ 19us) write_op.cc:276] Timestamp: P: 1765486517103885 usec, L: 0
1211 20:55:18.363250 (+ 164us) write_op.cc:270] Start()
1211 20:55:18.363269 (+ 19us) write_op.cc:276] Timestamp: P: 1765486517110129 usec, L: 0
1211 20:55:18.363408 (+ 139us) write_op.cc:270] Start()
1211 20:55:18.363427 (+ 19us) write_op.cc:276] Timestamp: P: 1765486517132846 usec, L: 0
1211 20:55:18.363574 (+ 147us) write_op.cc:270] Start()
1211 20:55:18.363594 (+ 20us) write_op.cc:276] Timestamp: P: 1765486517136873 usec, L: 0
1211 20:55:18.363755 (+ 161us) write_op.cc:270] Start()
1211 20:55:18.363773 (+ 18us) write_op.cc:276] Timestamp: P: 1765486517189610 usec, L: 0
1211 20:55:18.363936 (+ 163us) write_op.cc:270] Start()
1211 20:55:18.363956 (+ 20us) write_op.cc:276] Timestamp: P: 1765486517229784 usec, L: 0
1211 20:55:18.364110 (+ 154us) write_op.cc:270] Start()
1211 20:55:18.364141 (+ 31us) write_op.cc:276] Timestamp: P: 1765486517231010 usec, L: 0
1211 20:55:18.364273 (+ 132us) write_op.cc:270] Start()
1211 20:55:18.364293 (+ 20us) write_op.cc:276] Timestamp: P: 1765486517234263 usec, L: 0
1211 20:55:18.364516 (+ 223us) write_op.cc:270] Start()
1211 20:55:18.364536 (+ 20us) write_op.cc:276] Timestamp: P: 1765486517262925 usec, L: 0
1211 20:55:18.364679 (+ 143us) write_op.cc:270] Start()
1211 20:55:18.364698 (+ 19us) write_op.cc:276] Timestamp: P: 1765486517301397 usec, L: 0
1211 20:55:18.369077 (+ 4379us) write_op.cc:270] Start()
1211 20:55:18.369101 (+ 24us) write_op.cc:276] Timestamp: P: 1765486517330935 usec, L: 0
1211 20:55:18.382681 (+ 13580us) write_op.cc:270] Start()
1211 20:55:18.382703 (+ 22us) write_op.cc:276] Timestamp: P: 1765486517341045 usec, L: 0
1211 20:55:18.505417 (+122714us) log.cc:844] Serialized 114202 byte log entry
1211 20:55:18.510128 (+ 4711us) raft_consensus.cc:1645] Marking committed up to 1376
1211 20:55:18.667798 (+157670us) spinlock_profiling.cc:243] Waited 97.8 ms on lock 0x6160000c25b8. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fdcdf1055f0 00007fdcdf1d97da 00007fdcdf1d56c7 00007fdce304f21d 00007fdcdd1e8b46 00007fdcdd1e8875 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685 00007fdcda2cefaf
1211 20:55:18.667808 (+ 10us) raft_consensus.cc:1697] Filling consensus response to leader.
1211 20:55:18.667840 (+ 32us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
1211 20:55:18.667849 (+ 9us) raft_consensus.cc:1685] finished
1211 20:55:18.667851 (+ 2us) raft_consensus.cc:1691] UpdateReplicas() finished
1211 20:55:18.667928 (+ 77us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:55:18.103982 (+ 0us) write_op.cc:183] PREPARE: starting on tablet a9dbb37290964ca7ba650d2de1404245
1211 20:55:18.104064 (+ 82us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:55:18.104068 (+ 4us) write_op.cc:435] Acquired schema lock
1211 20:55:18.104070 (+ 2us) tablet.cc:660] Decoding operations
1211 20:55:18.104195 (+ 125us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:55:18.104207 (+ 12us) write_op.cc:641] Partition lock acquired for write op
1211 20:55:18.104211 (+ 4us) tablet.cc:683] Acquiring locks for 8 operations
1211 20:55:18.104290 (+ 79us) tablet.cc:699] Row locks acquired
1211 20:55:18.104292 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:55:18.510168 (+405876us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:18.104403 (+ 0us) write_op.cc:183] PREPARE: starting on tablet a9dbb37290964ca7ba650d2de1404245
1211 20:55:18.104500 (+ 97us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:55:18.104504 (+ 4us) write_op.cc:435] Acquired schema lock
1211 20:55:18.104506 (+ 2us) tablet.cc:660] Decoding operations
1211 20:55:18.104630 (+ 124us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:55:18.104642 (+ 12us) write_op.cc:641] Partition lock acquired for write op
1211 20:55:18.104646 (+ 4us) tablet.cc:683] Acquiring locks for 8 operations
1211 20:55:18.104736 (+ 90us) tablet.cc:699] Row locks acquired
1211 20:55:18.104739 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:55:18.510215 (+405476us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:18.104811 (+ 0us) write_op.cc:183] PREPARE: starting on tablet a9dbb37290964ca7ba650d2de1404245
1211 20:55:18.104879 (+ 68us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:55:18.104883 (+ 4us) write_op.cc:435] Acquired schema lock
1211 20:55:18.104885 (+ 2us) tablet.cc:660] Decoding operations
1211 20:55:18.105020 (+ 135us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:55:18.105030 (+ 10us) write_op.cc:641] Partition lock acquired for write op
1211 20:55:18.105033 (+ 3us) tablet.cc:683] Acquiring locks for 10 operations
1211 20:55:18.105107 (+ 74us) tablet.cc:699] Row locks acquired
1211 20:55:18.105110 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:55:18.510244 (+405134us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:18.105186 (+ 0us) write_op.cc:183] PREPARE: starting on tablet a9dbb37290964ca7ba650d2de1404245
1211 20:55:18.105254 (+ 68us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:55:18.105258 (+ 4us) write_op.cc:435] Acquired schema
I20251211 20:55:18.893934 7068 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:18.894771 6947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:42702 (request call id 9) took 1376 ms. Trace:
I20251211 20:55:18.912009 7068 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:18.912706 7068 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:18.917472 6947 rpcz_store.cc:276] 1211 20:55:17.518669 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:17.518816 (+ 147us) service_pool.cc:225] Handling call
1211 20:55:18.894541 (+1375725us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x61600001a254. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 00007fdce26306bd 00007fdce26307b3 00007fdcdf1f7521 00007fdce306129b 00007fdcdd1f1b07 00007fdcdd1f1835 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685 00007fdcda2cefaf 00007fdcda2cedd8
1211 20:55:18.894716 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3151238400}
I20251211 20:55:18.921788 6945 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.195:38469 (request call id 50) took 2318 ms. Trace:
I20251211 20:55:18.929332 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1334 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:18.929593 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1334 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:18.934343 6872 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:18.936606 7068 raft_consensus.cc:2749] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:19.332453 7068 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:19.333750 7068 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:19.334327 7051 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:19.334550 7051 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:19.335053 7068 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:19.335908 6274 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1420 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:19.336417 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1444 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:19.336671 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "a9dbb37290964ca7ba650d2de1404245" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1444 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:19.337256 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1420 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:19.338655 7051 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:19.351886 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:19.352205 6939 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.195:38469 (request call id 83) took 1005 ms. Trace:
I20251211 20:55:19.353426 7051 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:19.353592 6872 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:19.365016 7051 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:19.376013 7068 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:19.392155 7068 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:19.395141 7068 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:19.399813 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1334 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:19.408978 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1334 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:19.413566 6872 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:19.422493 7068 raft_consensus.cc:2749] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:19.372408 6939 rpcz_store.cc:276] 1211 20:55:18.327750 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:18.327808 (+ 58us) service_pool.cc:225] Handling call
1211 20:55:18.503236 (+175428us) raft_consensus.cc:1471] Updating replica for 167 ops
1211 20:55:18.574261 (+ 71025us) raft_consensus.cc:1534] Early marking committed up to index 1253
1211 20:55:18.574320 (+ 59us) raft_consensus.cc:1539] Triggering prepare for 167 ops
1211 20:55:18.574415 (+ 95us) write_op.cc:270] Start()
1211 20:55:18.574447 (+ 32us) write_op.cc:276] Timestamp: P: 1765486515104176 usec, L: 0
1211 20:55:18.574591 (+ 144us) write_op.cc:270] Start()
1211 20:55:18.574616 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515130271 usec, L: 0
1211 20:55:18.574803 (+ 187us) write_op.cc:270] Start()
1211 20:55:18.574824 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515156338 usec, L: 0
1211 20:55:18.574976 (+ 152us) write_op.cc:270] Start()
1211 20:55:18.574995 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515157397 usec, L: 0
1211 20:55:18.575157 (+ 162us) write_op.cc:270] Start()
1211 20:55:18.575176 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515199196 usec, L: 0
1211 20:55:18.575309 (+ 133us) write_op.cc:270] Start()
1211 20:55:18.575328 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515206446 usec, L: 0
1211 20:55:18.575466 (+ 138us) write_op.cc:270] Start()
1211 20:55:18.575509 (+ 43us) write_op.cc:276] Timestamp: P: 1765486515235681 usec, L: 0
1211 20:55:18.575655 (+ 146us) write_op.cc:270] Start()
1211 20:55:18.575673 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515240714 usec, L: 0
1211 20:55:18.575798 (+ 125us) write_op.cc:270] Start()
1211 20:55:18.575815 (+ 17us) write_op.cc:276] Timestamp: P: 1765486515264564 usec, L: 0
1211 20:55:18.575928 (+ 113us) write_op.cc:270] Start()
1211 20:55:18.575945 (+ 17us) write_op.cc:276] Timestamp: P: 1765486515273885 usec, L: 0
1211 20:55:18.576060 (+ 115us) write_op.cc:270] Start()
1211 20:55:18.576076 (+ 16us) write_op.cc:276] Timestamp: P: 1765486515310363 usec, L: 0
1211 20:55:18.576194 (+ 118us) write_op.cc:270] Start()
1211 20:55:18.576209 (+ 15us) write_op.cc:276] Timestamp: P: 1765486515325602 usec, L: 0
1211 20:55:18.576344 (+ 135us) write_op.cc:270] Start()
1211 20:55:18.576361 (+ 17us) write_op.cc:276] Timestamp: P: 1765486515333871 usec, L: 0
1211 20:55:18.576516 (+ 155us) write_op.cc:270] Start()
1211 20:55:18.576535 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515343355 usec, L: 0
1211 20:55:18.576660 (+ 125us) write_op.cc:270] Start()
1211 20:55:18.576678 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515380680 usec, L: 0
1211 20:55:18.578035 (+ 1357us) write_op.cc:270] Start()
1211 20:55:18.578057 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515384708 usec, L: 0
1211 20:55:18.595803 (+ 17746us) write_op.cc:270] Start()
1211 20:55:18.595824 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515394241 usec, L: 0
1211 20:55:18.602899 (+ 7075us) write_op.cc:270] Start()
1211 20:55:18.602925 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515413877 usec, L: 0
1211 20:55:18.603969 (+ 1044us) write_op.cc:270] Start()
1211 20:55:18.603996 (+ 27us) write_op.cc:276] Timestamp: P: 1765486515442839 usec, L: 0
1211 20:55:18.604144 (+ 148us) write_op.cc:270] Start()
1211 20:55:18.604164 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515460894 usec, L: 0
1211 20:55:18.607444 (+ 3280us) write_op.cc:270] Start()
1211 20:55:18.607469 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515468586 usec, L: 0
1211 20:55:18.607624 (+ 155us) write_op.cc:270] Start()
1211 20:55:18.607647 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515483133 usec, L: 0
1211 20:55:18.607835 (+ 188us) write_op.cc:270] Start()
1211 20:55:18.607856 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515496343 usec, L: 0
1211 20:55:18.608023 (+ 167us) write_op.cc:270] Start()
1211 20:55:18.608043 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515506129 usec, L: 0
1211 20:55:18.608217 (+ 174us) write_op.cc:270] Start()
1211 20:55:18.608245 (+ 28us) write_op.cc:276] Timestamp: P: 1765486515516043 usec, L: 0
1211 20:55:18.608360 (+ 115us) write_op.cc:270] Start()
1211 20:55:18.608391 (+ 31us) write_op.cc:276] Timestamp: P: 1765486515547539 usec, L: 0
1211 20:55:18.608529 (+ 138us) write_op.cc:270] Start()
1211 20:55:18.608547 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515561619 usec, L: 0
1211 20:55:18.608670 (+ 123us) write_op.cc:270] Start()
1211 20:55:18.608688 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515562840 usec, L: 0
1211 20:55:18.608801 (+ 113us) write_op.cc:270] Start()
1211 20:55:18.608822 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515588769 usec, L: 0
1211 20:55:18.608970 (+ 148us) write_op.cc:270] Start()
1211 20:55:18.608991 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515592935 usec, L: 0
1211 20:55:18.610406 (+ 1415us) write_op.cc:270] Start()
1211 20:55:18.610430 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515634612 usec, L: 0
1211 20:55:18.612770 (+ 2340us) write_op.cc:270] Start()
1211 20:55:18.612794 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515645269 usec, L: 0
1211 20:55:18.633387 (+ 20593us) write_op.cc:270] Start()
1211 20:55:18.633419 (+ 32us) write_op.cc:276] Timestamp: P: 1765486515646102 usec, L: 0
1211 20:55:18.633618 (+ 199us) write_op.cc:270] Start()
1211 20:55:18.633644 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515646727 usec, L: 0
1211 20:55:18.633862 (+ 218us) write_op.cc:270] Start()
1211 20:55:18.633886 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515675201 usec, L: 0
1211 20:55:18.634074 (+ 188us) write_op.cc:270] Start()
1211 20:55:18.634098 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515693107 usec, L: 0
1211 20:55:18.634281 (+ 183us) write_op.cc:270] Start()
1211 20:55:18.634307 (+ 26us) write_op.cc:276] Timestamp: P: 1765486515711465 usec, L: 0
1211 20:55:18.634485 (+ 178us) write_op.cc:270] Start()
1211 20:55:18.634508 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515726846 usec, L: 0
1211 20:55:18.672739 (+ 38231us) write_op.cc:270] Start()
1211 20:55:18.672762 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515746754 usec, L: 0
1211 20:55:18.683190 (+ 10428us) write_op.cc:270] Start()
1211 20:55:18.683208 (+ 18us) write_op.cc:276] Timestamp: P: 1765486515761736 usec, L: 0
1211 20:55:18.683343 (+ 135us) write_op.cc:270] Start()
1211 20:55:18.683356 (+ 13us) write_op.cc:276] Timestamp: P: 1765486515774029 usec, L: 0
1211 20:55:18.685195 (+ 1839us) write_op.cc:270] Start()
1211 20:55:18.685214 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515781258 usec, L: 0
1211 20:55:18.690720 (+ 5506us) write_op.cc:270] Start()
1211 20:55:18.690745 (+ 25us) write_op.cc:276] Timestamp: P: 1765486515804145 usec, L: 0
1211 20:55:18.696430 (+ 5685us) write_op.cc:270] Start()
1211 20:55:18.696454 (+ 24us) write_op.cc:276] Timestamp: P: 1765486515824432 usec, L: 0
1211 20:55:18.711394 (+ 14940us) write_op.cc:270] Start()
1211 20:55:18.711416 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515841925 usec, L: 0
1211 20:55:18.722354 (+ 10938us) write_op.cc:270] Start()
1211 20:55:18.722382 (+ 28us) write_op.cc:276] Timestamp: P: 1765486515865355 usec, L: 0
1211 20:55:18.722563 (+ 181us) write_op.cc:270] Start()
1211 20:55:18.722585 (+ 22us) write_op.cc:276] Timestamp: P: 1765486515868042 usec, L: 0
1211 20:55:18.722743 (+ 158us) write_op.cc:270] Start()
1211 20:55:18.722764 (+ 21us) write_op.cc:276] Timestamp: P: 1765486515879965 usec, L: 0
1211 20:55:18.722904 (+ 140us) write_op.cc:270] Start()
1211 20:55:18.722923 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515927814 usec, L: 0
1211 20:55:18.723059 (+ 136us) write_op.cc:270] Start()
1211 20:55:18.723078 (+ 19us) write_op.cc:276] Timestamp: P: 1765486515936019 usec, L: 0
1211 20:55:18.723654 (+ 576us) write_op.cc:270] Start()
1211 20:55:18.723677 (+ 23us) write_op.cc:276] Timestamp: P: 1765486515953731 usec, L: 0
1211 20:55:18.723849 (+ 172us) write_op.cc:270] Start()
1211 20:55:18.723869 (+ 20us) write_op.cc:276] Timestamp: P: 1765486515987327 usec, L: 0
1211 20:55:18.724019 (+ 150us) write_op.cc:270] Start()
1211 20:55:18.724037 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516008729 usec, L: 0
1211 20:55:18.724190 (+ 153us) write_op.cc:270] Start()
1211 20:55:18.724212 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516040041 usec, L: 0
1211 20:55:18.742576 (+ 18364us) write_op.cc:270] Start()
1211 20:55:18.742757 (+ 181us) write_op.cc:276] Timestamp: P: 1765486516040630 usec, L: 0
1211 20:55:18.754491 (+ 11734us) write_op.cc:270] Start()
1211 20:55:18.754517 (+ 26us) write_op.cc:276] Timestamp: P: 1765486516057664 usec, L: 0
1211 20:55:18.754665 (+ 148us) write_op.cc:270] Start()
1211 20:55:18.754683 (+ 18us) write_op.cc:276] Timestamp: P: 1765486516069026 usec, L: 0
1211 20:55:18.754954 (+ 271us) write_op.cc:270] Start()
1211 20:55:18.754979 (+ 25us) write_op.cc:276] Timestamp: P: 1765486516096219 usec, L: 0
1211 20:55:18.755137 (+ 158us) write_op.cc:270] Start()
1211 20:55:18.755159 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516100274 usec, L: 0
1211 20:55:18.755290 (+ 131us) write_op.cc:270] Start()
1211 20:55:18.755328 (+ 38us) write_op.cc:276] Timestamp: P: 1765486516113770 usec, L: 0
1211 20:55:18.755436 (+ 108us) write_op.cc:270] Start()
1211 20:55:18.755455 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516129038 usec, L: 0
1211 20:55:18.755748 (+ 293us) write_op.cc:270] Start()
1211 20:55:18.755767 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516167752 usec, L: 0
1211 20:55:18.757470 (+ 1703us) write_op.cc:270] Start()
1211 20:55:18.757496 (+ 26us) write_op.cc:276] Timestamp: P: 1765486516173679 usec, L: 0
1211 20:55:18.757671 (+ 175us) write_op.cc:270] Start()
1211 20:55:18.757692 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516178673 usec, L: 0
1211 20:55:18.757822 (+ 130us) write_op.cc:270] Start()
1211 20:55:18.757843 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516193518 usec, L: 0
1211 20:55:18.758011 (+ 168us) write_op.cc:270] Start()
1211 20:55:18.758034 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516214506 usec, L: 0
1211 20:55:18.759789 (+ 1755us) write_op.cc:270] Start()
1211 20:55:18.759812 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516250636 usec, L: 0
1211 20:55:18.759967 (+ 155us) write_op.cc:270] Start()
1211 20:55:18.759988 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516259972 usec, L: 0
1211 20:55:18.760145 (+ 157us) write_op.cc:270] Start()
1211 20:55:18.760178 (+ 33us) write_op.cc:276] Timestamp: P: 1765486516261329 usec, L: 0
1211 20:55:18.760348 (+ 170us) write_op.cc:270] Start()
1211 20:55:18.760369 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516281512 usec, L: 0
1211 20:55:18.760532 (+ 163us) write_op.cc:270] Start()
1211 20:55:18.760551 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516340236 usec, L: 0
1211 20:55:18.761749 (+ 1198us) write_op.cc:270] Start()
1211 20:55:18.761772 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516347529 usec, L: 0
1211 20:55:18.761917 (+ 145us) write_op.cc:270] Start()
1211 20:55:18.761938 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516349350 usec, L: 0
1211 20:55:18.762569 (+ 631us) write_op.cc:270] Start()
1211 20:55:18.762591 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516381272 usec, L: 0
1211 20:55:18.766722 (+ 4131us) write_op.cc:270] Start()
1211 20:55:18.766747 (+ 25us) write_op.cc:276] Timestamp: P: 1765486516401129 usec, L: 0
1211 20:55:18.775736 (+ 8989us) write_op.cc:270] Start()
1211 20:55:18.780283 (+ 4547us) write_op.cc:276] Timestamp: P: 1765486516417642 usec, L: 0
1211 20:55:18.780484 (+ 201us) write_op.cc:270] Start()
1211 20:55:18.780507 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516447809 usec, L: 0
1211 20:55:18.780678 (+ 171us) write_op.cc:270] Start()
1211 20:55:18.780712 (+ 34us) write_op.cc:276] Timestamp: P: 1765486516473516 usec, L: 0
1211 20:55:18.780870 (+ 158us) write_op.cc:270] Start()
1211 20:55:18.781404 (+ 534us) write_op.cc:276] Timestamp: P: 1765486516488199 usec, L: 0
1211 20:55:18.781754 (+ 350us) write_op.cc:270] Start()
1211 20:55:18.781777 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516509290 usec, L: 0
1211 20:55:18.781933 (+ 156us) write_op.cc:270] Start()
1211 20:55:18.781954 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516516800 usec, L: 0
1211 20:55:18.782965 (+ 1011us) write_op.cc:270] Start()
1211 20:55:18.782987 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516557344 usec, L: 0
1211 20:55:18.783731 (+ 744us) write_op.cc:270] Start()
1211 20:55:18.783756 (+ 25us) write_op.cc:276] Timestamp: P: 1765486516579293 usec, L: 0
1211 20:55:18.783910 (+ 154us) write_op.cc:270] Start()
1211 20:55:18.783932 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516586570 usec, L: 0
1211 20:55:18.784114 (+ 182us) write_op.cc:270] Start()
1211 20:55:18.784133 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516590360 usec, L: 0
1211 20:55:18.784288 (+ 155us) write_op.cc:270] Start()
1211 20:55:18.784308 (+ 20us) write_op.cc:276] Timestamp: P: 1765486516631446 usec, L: 0
1211 20:55:18.784483 (+ 175us) write_op.cc:270] Start()
1211 20:55:18.784502 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516657848 usec, L: 0
1211 20:55:18.784675 (+ 173us) write_op.cc:270] Start()
1211 20:55:18.784694 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516671398 usec, L: 0
1211 20:55:18.784840 (+ 146us) write_op.cc:270] Start()
1211 20:55:18.784859 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516680281 usec, L: 0
1211 20:55:18.786650 (+ 1791us) write_op.cc:270] Start()
1211 20:55:18.786974 (+ 324us) write_op.cc:276] Timestamp: P: 1765486516688618 usec, L: 0
1211 20:55:18.791175 (+ 4201us) write_op.cc:270] Start()
1211 20:55:18.791198 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516713209 usec, L: 0
1211 20:55:18.796910 (+ 5712us) write_op.cc:270] Start()
1211 20:55:18.796933 (+ 23us) write_op.cc:276] Timestamp: P: 1765486516727792 usec, L: 0
1211 20:55:18.819530 (+ 22597us) write_op.cc:270] Start()
1211 20:55:18.819552 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516755400 usec, L: 0
1211 20:55:18.827886 (+ 8334us) write_op.cc:270] Start()
1211 20:55:18.827907 (+ 21us) write_op.cc:276] Timestamp: P: 1765486516773436 usec, L: 0
1211 20:55:18.828070 (+ 163us) write_op.cc:270] Start()
1211 20:55:18.828092 (+ 22us) write_op.cc:276] Timestamp: P: 1765486516783306 usec, L: 0
1211 20:55:18.830082 (+ 1990us) write_op.cc:270] Start()
1211 20:55:18.830102 (+ 20us) write_op.cc:276] Timestamp: P: 1765486516794079 usec, L: 0
1211 20:55:18.832308 (+ 2206us) write_op.cc:270] Start()
1211 20:55:18.832332 (+ 24us) write_op.cc:276] Timestamp: P: 1765486516822726 usec, L: 0
1211 20:55:18.832515 (+ 183us) write_op.cc:270] Start()
1211 20:55:18.832534 (+ 19us) write_op.cc:276] Timestamp: P: 1765486516823381 usec, L: 0
1211 20:55:18.832654 (+ 120us) write_op.cc:270] Start()
1211 20:55:18.832670 (+ 16us) write_op.cc:276] Timestamp: P: 1765486516886168 usec, L: 0
1211 20:55:18.832777 (+ 107us) write_op.cc:270] Start()
1211 20:55:18.832792 (+ 15us) write_op.cc:276] Timestamp: P: 1765486516896127 usec, L: 0
1211 20:55:18.832927 (+ 135us) write_op.cc:270] Start()
1211 20:55:18.832944 (+ 17us) write_op.cc:276] Timestamp: P: 1765486516924980 usec, L: 0
1211 20:55:18.833054 (+ 110us) write_op.cc:270] Start()
1211 20:55:18.833069 (+ 15us) write_op.cc:276] Timestamp: P: 1765486516925860 usec, L: 0
1211 20:55:18.833180 (+ 111us) write_op.cc:270] Start()
1211 20:55:18.833195 (+ 15us) write_op.cc:276] Timestamp: P: 1765486516926498 usec, L: 0
1211 20:55:18.833382 (+ 187us) write_op.cc:270] Start()
1211 20:55:18.833399 (+ 17us) write_op.cc:276] Timestamp: P: 1765486516966217 usec, L: 0
1211 20:55:18.833523 (+ 124us) write_op.cc:270] Start()
1211 20:55:18.833539 (+ 16us) write_op.cc:276] Timestamp: P: 1765486516990172 usec, L: 0
1211 20:55:18.833657 (+ 118us) write_op.cc:270] Start()
1211 20:55:18.833672 (+ 15us) write_op.cc:276] Timestamp: P: 1765486516990776 usec, L: 0
1211 20:55:18.833787 (+ 115us) write_op.cc:270] Start()
1211 20:55:18.833803 (+ 16us) write_op.cc:276] Timestamp: P: 1765486516992402 usec, L: 0
1211 20:55:18.833920 (+ 117us) write_op.cc:270] Start()
1211 20:55:18.833940 (+ 20us) write_op.cc:276] Timestamp: P: 1765486517011383 usec, L: 0
1211 20:55:18.834119 (+ 179us) write_op.cc:270] Start()
1211 20:55:18.834141 (+ 22us) write_op.cc:276] Timestamp: P: 1765486517039210 usec, L: 0
1211 20:55:18.834308 (+ 167us) write_op.cc:270] Start()
1211 20:55:18.834326 (+ 18us) write_op.cc:276] Timestamp: P: 1765486517052412 usec, L: 0
1211 20:55:18.834493 (+ 167us) write_op.cc:270] Start()
1211 20:55:18.834518 (+ 25us) write_op.cc:276] Timestamp: P: 1765486517075068 usec, L: 0
1211 20:55:18.865927 (+ 31409us) write_op.cc:270] Start()
1211 20:55:18.865956 (+ 29us) write_op.cc:276] Timestamp: P: 1765486517106969 usec, L: 0
1211 20:55:18.875675 (+ 9719us) write_op.cc:270] Start()
1211 20:55:18.875695 (+ 20us) write_op.cc:276] Timestamp: P: 1765486517115274 usec, L: 0
1211 20:55:18.875873 (+ 178us) write_op.cc:270] Start()
1211 20:55:18.890091 (+ 14218us) write_op.cc:276] Timestamp: P: 1765486517138021 usec, L: 0
1211 20:55:18.903475 (+ 13384us) write_op.cc:270] Start()
1211 20:55:18.903499 (+ 24us) write_op.cc:276] Timestamp: P: 1765486517138617 usec, L: 0
1211 20:55:18.903917 (+ 418us) write_op.cc:270] Start()
1211 20:55:18.903943 (+ 26us) write_op.cc:276] Timestamp: P: 1765486517191039 usec, L: 0
1211 20:55:18.904109 (+ 166us) write_op.cc:270] Start()
1211 20:55:18.904670 (+ 561us) write_op.cc:276] Timestamp: P: 1765486517231774 usec, L: 0
1211 20:55:18.904834 (+ 164us) write_op.cc:270] Start()
1211 20:55:18.904860 (+ 26us) write_op.cc:276] Timestamp: P: 1765486517233281 usec, L: 0
1211 20:55:18.905013 (+ 153us) write_op.cc:270] Start()
1211 20:55:18.905053 (+ 40us) write_op.cc:276] Timestamp: P: 1765486517235252 usec, L: 0
1211 20:55:18.905246 (+ 193us) write_op.cc:270] Start()
1211 20:55:18.905266 (+ 20us) write_op.cc:276] Timestamp: P: 1765486517265820 usec, L: 0
1211 20:55:18.905463 (+ 197us) write_op.cc:270] Start()
1211 20:55:18.905485 (+ 22us) write_op.cc:276] Timestamp: P: 1765486517309236 usec, L: 0
1211 20:55:18.906079 (+ 594us) write_op.cc:270] Start()
1211 20:55:18.906103 (+ 24us) write_op.cc:276] Timestamp: P: 1765486517338298 usec, L: 0
1211 20:55:18.912035 (+ 5932us) write_op.cc:270] Start()
1211 20:55:18.912062 (+ 27us) write_op.cc:276] Timestamp: P: 1765486517349038 usec, L: 0
1211 20:55:18.912235 (+ 173us) write_op.cc:270] Start()
1211 20:55:18.912257 (+ 22us) write_op.cc:276] Timestamp: P: 1765486517386399 usec, L: 0
1211 20:55:18.912410 (+ 153us) write_op.cc:270] Start()
1211 20:55:18.912429 (+ 19us) write_op.cc:276] Timestamp: P: 1765486517410961 usec, L: 0
1211 20:55:18.912574 (+ 145us) write_op.cc:270] Start()
1211 20:55:18.912594 (+ 20us) write_op.cc:276] Timestamp: P: 1765486517436173 usec, L: 0
1211 20:55:18.912748 (+ 154us) write_op.cc:270] Start()
1211 20:55:18.912770 (+ 22us) write_op.cc:276] Timestamp: P: 1765486517470518 usec, L: 0
1211 20:55:18.920904 (+ 8134us) write_op.cc:270] Start()
1211 20:55:18.920930 (+ 26us) write_op.cc:276] Timestamp: P: 1765486517471133 usec, L: 0
1211 20:55:18.946307 (+ 25377us) write_op.cc:270] Start()
1211 20:55:18.946335 (+ 28us) write_op.cc:276] Timestamp: P: 1765486517479684 usec, L: 0
1211 20:55:18.959902 (+ 13567us) write_op.cc:270] Start()
1211 20:55:18.982208 (+ 22306us) write_op.cc:276] Timestamp: P: 1765486517491467 usec, L: 0
1211 20:55:18.993062 (+ 10854us) write_op.cc:270] Start()
1211 20:55:19.003454 (+ 10392us) write_op.cc:276] Timestamp: P: 1765486517514985 usec, L: 0
1211 20:55:19.003671 (+ 217us) write_op.cc:270] Start()
1211 20:55:19.003699 (+ 28us) write_op.cc:276] Timestamp: P: 1765486517545230 usec, L: 0
1211 20:55:19.003870 (+ 171us) write_op.cc:270] Start()
1211 20:55:19.003893 (+ 23us) write_op.cc:276] Timestamp: P: 1765486517547149 usec, L: 0
1211 20:55:19.004434 (+ 541us) write_op.cc:270] Start()
1211 20:55:19.004462 (+ 28us) write_op.cc:276] Timestamp: P: 1765486517576699 usec, L: 0
1211 20:55:19.004639 (+ 177us) write_op.cc:270] Start()
1211 20:55:19.004664 (+ 25us) write_op.cc:276] Timestamp: P: 1765486517577377 usec, L: 0
1211 20:55:19.004838 (+ 174us) write_op.cc:270] Start()
1211 20:55:19.004860 (+ 22us) write_op.cc:276] Timestamp: P: 1765486517608641 usec, L: 0
1211 20:55:19.005029 (+ 169us) write_op.cc:270] Start()
1211 20:55:19.005059 (+ 30us) write_op.cc:276] Timestamp: P: 1765486517612811 usec, L: 0
1211 20:55:19.005219 (+ 160us) write_op.cc:270] Start()
1211 20:55:19.005243 (+ 24us) write_op.cc:276] Timestamp: P: 1765486517653728 usec, L: 0
1211 20:55:19.005417 (+ 174us) write_op.cc:270] Start()
1211 20:55:19.005440 (+ 23us) write_op.cc:276] Timestamp: P: 1765486517660508 usec, L: 0
1211 20:55:19.005619 (+ 179us) write_op.cc:270] Start()
1211 20:55:19.005642 (+ 23us) write_op.cc:276] Timestamp: P: 1765486517681048 usec, L: 0
1211 20:55:19.005880 (+ 238us) write_op.cc:270] Start()
1211 20:55:19.005904 (+ 24us) write_op.cc:276] Timestamp: P: 1765486517681983 usec, L: 0
1211 20:55:19.006086 (+ 182us) write_op.cc:270] Start()
1211 20:55:19.006110 (+ 24us) write_op.cc:276] Timestamp: P: 1765486517732861 usec, L: 0
1211 20:55:19.006267 (+ 157us) write_op.cc:270] Start()
1211 20:55:19.006288 (+ 21us) write_op.cc:276] Timestamp: P: 1765486517734417 usec, L: 0
1211 20:55:19.006482 (+ 194us) write_op.cc:270] Start()
1211 20:55:19.006503 (+ 21us) write_op.cc:276] Timestamp: P: 1765486517735949 usec, L: 0
1211 20:55:19.006722 (+ 219us) write_op.cc:270] Start()
1211 20:55:19.006744 (+ 22us) write_op.cc:276] Timestamp: P: 1765486517738350 usec, L: 0
1211 20:55:19.006887 (+ 143us) write_op.cc:270] Start()
1211 20:55:19.006924 (+ 37us) write_op.cc:276] Timestamp: P: 1765486517782790 usec, L: 0
1211 20:55:19.010927 (+ 4003us) write_op.cc:270] Start()
1211 20:55:19.010951 (+ 24us) write_op.cc:276] Timestamp: P: 1765486517790528 usec, L: 0
1211 20:55:19.044337 (+ 33386us) write_op.cc:270] Start()
1211 20:55:19.044372 (+ 35us) write_op.cc:276] Timestamp: P: 1765486517797082 usec, L: 0
1211 20:55:19.045234 (+ 862us) write_op.cc:270] Start()
1211 20:55:19.045253 (+ 19us) write_op.cc:276] Timestamp: P: 1765486517813346 usec, L: 0
1211 20:55:19.062138 (+ 16885us) write_op.cc:270] Start()
1211 20:55:19.062163 (+ 25us) write_op.cc:276] Timestamp: P: 1765486517872324 usec, L: 0
1211 20:55:19.097299 (+ 35136us) write_op.cc:270] Start()
1211 20:55:19.097329 (+ 30us) write_op.cc:276] Timestamp: P: 1765486517879395 usec, L: 0
1211 20:55:19.097516 (+ 187us) write_op.cc:270] Start()
1211 20:55:19.097538 (+ 22us) write_op.cc:276] Timestamp: P: 1765486517899309 usec, L: 0
1211 20:55:19.103018 (+ 5480us) write_op.cc:270] Start()
1211 20:55:19.103047 (+ 29us) write_op.cc:276] Timestamp: P: 1765486517952640 usec, L: 0
1211 20:55:19.103234 (+ 187us) write_op.cc:270] Start()
1211 20:55:19.103278 (+ 44us) write_op.cc:276] Timestamp: P: 1765486517967647 usec, L: 0
1211 20:55:19.106915 (+ 3637us) write_op.cc:270] Start()
1211 20:55:19.106936 (+ 21us) write_op.cc:276] Timestamp: P: 1765486517986578 usec, L: 0
1211 20:55:19.109560 (+ 2624us) write_op.cc:270] Start()
1211 20:55:19.109585 (+ 25us) write_op.cc:276] Timestamp: P: 1765486518011670 usec, L: 0
1211 20:55:19.109758 (+ 173us) write_op.cc:270] Start()
1211 20:55:19.109780 (+ 22us) write_op.cc:276] Timestamp: P: 1765486518032238 usec, L: 0
1211 20:55:19.111766 (+ 1986us) write_op.cc:270] Start()
1211 20:55:19.111808 (+ 42us) write_op.cc:276] Timestamp: P: 1765486518062556 usec, L: 0
1211 20:55:19.111989 (+ 181us) write_op.cc:270] Start()
1211 20:55:19.112014 (+ 25us) write_op.cc:276] Timestamp: P: 1765486518083934 usec, L: 0
1211 20:55:19.112150 (+ 136us) write_op.cc:270] Start()
1211 20:55:19.112169 (+ 19us) write_op.cc:276] Timestamp: P: 1765486518095512 usec, L: 0
1211 20:55:19.112510 (+ 341us) write_op.cc:270] Start()
1211 20:55:19.112531 (+ 21us) write_op.cc:276] Timestamp: P: 1765486518108704 usec, L: 0
1211 20:55:19.112672 (+ 141us) write_op.cc:270] Start()
1211 20:55:19.112714 (+ 42us) write_op.cc:276] Timestamp: P: 1765486518156147 usec, L: 0
1211 20:55:19.112854 (+ 140us) write_op.cc:270] Start()
1211 20:55:19.112873 (+ 19us) write_op.cc:276] Timestamp: P: 1765486518166865 usec, L: 0
1211 20:55:19.116902 (+ 4029us) write_op.cc:270] Start()
1211 20:55:19.116924 (+ 22us) write_op.cc:276] Timestamp: P: 1765486518167798 usec, L: 0
1211 20:55:19.117065 (+ 141us) write_op.cc:270] Start()
1211 20:55:19.117086 (+ 21us) write_op.cc:276] Timestamp: P: 1765486518180648 usec, L: 0
1211 20:55:19.117822 (+ 736us) write_op.cc:270] Start()
1211 20:55:19.117843 (+ 21us) write_op.cc:276] Timestamp: P: 1765486518268896 usec, L: 0
1211 20:55:19.121370 (+ 3527us) write_op.cc:270] Start()
1211 20:55:19.121384 (+ 14us) write_op.cc:276] Timestamp: P: 1765486518269671 usec, L: 0
1211 20:55:19.177602 (+ 56218us) log.cc:844] Serialized 112207 byte log entry
1211 20:55:19.177747 (+ 145us) raft_consensus.cc:1645] Marking committed up to 1420
1211 20:55:19.333512 (+155765us) raft_consensus.cc:1697] Filling consensus response to leader.
1211 20:55:19.333548 (+ 36us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
1211 20:55:19.333559 (+ 11us) raft_consensus.cc:1685] finished
1211 20:55:19.333563 (+ 4us) raft_consensus.cc:1691] UpdateReplicas() finished
1211 20:55:19.333644 (+ 81us) inbound_call.cc:173] Queueing success response
Related trace 'op':
1211 20:55:19.067329 (+ 0us) write_op.cc:183] PREPARE: starting on tablet ce1e22ac4b814a508b20688b15d0c959
1211 20:55:19.067432 (+ 103us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:55:19.067438 (+ 6us) write_op.cc:435] Acquired schema lock
1211 20:55:19.067440 (+ 2us) tablet.cc:660] Decoding operations
1211 20:55:19.067527 (+ 87us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:55:19.067538 (+ 11us) write_op.cc:641] Partition lock acquired for write op
1211 20:55:19.067543 (+ 5us) tablet.cc:683] Acquiring locks for 4 operations
1211 20:55:19.067616 (+ 73us) tablet.cc:699] Row locks acquired
1211 20:55:19.067618 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:55:19.177804 (+110186us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:19.068202 (+ 0us) write_op.cc:183] PREPARE: starting on tablet ce1e22ac4b814a508b20688b15d0c959
1211 20:55:19.068307 (+ 105us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:55:19.068311 (+ 4us) write_op.cc:435] Acquired schema lock
1211 20:55:19.068313 (+ 2us) tablet.cc:660] Decoding operations
1211 20:55:19.068475 (+ 162us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:55:19.068486 (+ 11us) write_op.cc:641] Partition lock acquired for write op
1211 20:55:19.068496 (+ 10us) tablet.cc:683] Acquiring locks for 7 operations
1211 20:55:19.068566 (+ 70us) tablet.cc:699] Row locks acquired
1211 20:55:19.068568 (+ 2us) write_op.cc:260] PREPARE: finished
1211 20:55:19.177851 (+109283us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:19.068651 (+ 0us) write_op.cc:183] PREPARE: starting on tablet ce1e22ac4b814a508b20688b15d0c959
1211 20:55:19.068742 (+ 91us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:55:19.068746 (+ 4us) write_op.cc:435] Acquired schema lock
1211 20:55:19.068748 (+ 2us) tablet.cc:660] Decoding operations
1211 20:55:19.068878 (+ 130us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:55:19.068889 (+ 11us) write_op.cc:641] Partition lock acquired for write op
1211 20:55:19.068893 (+ 4us) tablet.cc:683] Acquiring locks for 9 operations
1211 20:55:19.068985 (+ 92us) tablet.cc:699] Row locks acquired
1211 20:55:19.068988 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:55:19.177879 (+108891us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:19.069069 (+ 0us) write_op.cc:183] PREPARE: starting on tablet ce1e22ac4b814a508b20688b15d0c959
1211 20:55:19.069168 (+ 99us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:55:19.069173 (+ 5us) write_op.cc:435] Acquired schema lock
1211 20:55:19.069175 (+ 2us) tablet.cc:660] Decoding operations
1211 20:55:19.069298 (+ 123us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:55:19.069310 (+ 12us) write_op.cc:641] Partition lock acquired for write op
1211 20:55:19.069315 (+ 5us) tablet.cc:683] Acquiring locks for 8 operations
1211 20:55:19.069417 (+ 102us) tablet.cc:699] Row locks acquired
1211 20:55:19.069420 (+ 3us) write_op.cc:260] PREPARE: finished
1211 20:55:19.177905 (+108485us) op_driver.cc:464] REPLICATION: finished
Related trace 'op':
1211 20:55:19.069515 (+ 0us) write_op.cc:183] PREPARE: starting on tablet ce1e22ac4b814a508b20688b15d0c959
1211 20:55:19.079709 (+ 10194us) write_op.cc:432] Acquiring schema lock in shared mode
1211 20:55:19.079713 (+ 4us) write_op.cc:435] Acquired schema lock
1211 20:55:19.079714 (+ 1us) tablet.cc:660] Decoding operations
1211 20:55:19.095293 (+ 15579us) write_op.cc:620] Acquiring the partition lock for write op
1211 20:55:19.095305 (+ 12us) write_op.cc:641] Partition lock acquired for write op
1211 20:55:19.095308 (+ 3us) tablet.cc:683] Acqu
I20251211 20:55:18.936924 6945 rpcz_store.cc:276] 1211 20:55:16.571148 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:16.571202 (+ 54us) service_pool.cc:225] Handling call
1211 20:55:16.728567 (+157365us) raft_consensus.cc:1471] Updating replica for 348 ops
1211 20:55:17.069093 (+340526us) raft_consensus.cc:1534] Early marking committed up to index 986
1211 20:55:17.069159 (+ 66us) raft_consensus.cc:1539] Triggering prepare for 348 ops
1211 20:55:17.069260 (+ 101us) write_op.cc:270] Start()
1211 20:55:17.069283 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510100376 usec, L: 0
1211 20:55:17.069527 (+ 244us) write_op.cc:270] Start()
1211 20:55:17.069553 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510146467 usec, L: 0
1211 20:55:17.069766 (+ 213us) write_op.cc:270] Start()
1211 20:55:17.069787 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510155700 usec, L: 0
1211 20:55:17.069982 (+ 195us) write_op.cc:270] Start()
1211 20:55:17.070004 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510160836 usec, L: 0
1211 20:55:17.070249 (+ 245us) write_op.cc:270] Start()
1211 20:55:17.070271 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510201427 usec, L: 0
1211 20:55:17.070452 (+ 181us) write_op.cc:270] Start()
1211 20:55:17.070472 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510208934 usec, L: 0
1211 20:55:17.070673 (+ 201us) write_op.cc:270] Start()
1211 20:55:17.070717 (+ 44us) write_op.cc:276] Timestamp: P: 1765486510228828 usec, L: 0
1211 20:55:17.070932 (+ 215us) write_op.cc:270] Start()
1211 20:55:17.070966 (+ 34us) write_op.cc:276] Timestamp: P: 1765486510235288 usec, L: 0
1211 20:55:17.071187 (+ 221us) write_op.cc:270] Start()
1211 20:55:17.071208 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510279542 usec, L: 0
1211 20:55:17.071412 (+ 204us) write_op.cc:270] Start()
1211 20:55:17.071433 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510311620 usec, L: 0
1211 20:55:17.071704 (+ 271us) write_op.cc:270] Start()
1211 20:55:17.071727 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510328479 usec, L: 0
1211 20:55:17.071909 (+ 182us) write_op.cc:270] Start()
1211 20:55:17.071929 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510362448 usec, L: 0
1211 20:55:17.087877 (+ 15948us) write_op.cc:270] Start()
1211 20:55:17.087904 (+ 27us) write_op.cc:276] Timestamp: P: 1765486510385550 usec, L: 0
1211 20:55:17.104638 (+ 16734us) write_op.cc:270] Start()
1211 20:55:17.104662 (+ 24us) write_op.cc:276] Timestamp: P: 1765486510390400 usec, L: 0
1211 20:55:17.104822 (+ 160us) write_op.cc:270] Start()
1211 20:55:17.104844 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510432801 usec, L: 0
1211 20:55:17.104985 (+ 141us) write_op.cc:270] Start()
1211 20:55:17.105004 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510436048 usec, L: 0
1211 20:55:17.105141 (+ 137us) write_op.cc:270] Start()
1211 20:55:17.105160 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510483899 usec, L: 0
1211 20:55:17.105291 (+ 131us) write_op.cc:270] Start()
1211 20:55:17.105318 (+ 27us) write_op.cc:276] Timestamp: P: 1765486510484844 usec, L: 0
1211 20:55:17.105479 (+ 161us) write_op.cc:270] Start()
1211 20:55:17.105500 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510488402 usec, L: 0
1211 20:55:17.105668 (+ 168us) write_op.cc:270] Start()
1211 20:55:17.105690 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510518580 usec, L: 0
1211 20:55:17.130956 (+ 25266us) write_op.cc:270] Start()
1211 20:55:17.130981 (+ 25us) write_op.cc:276] Timestamp: P: 1765486510553916 usec, L: 0
1211 20:55:17.131138 (+ 157us) write_op.cc:270] Start()
1211 20:55:17.131159 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510562171 usec, L: 0
1211 20:55:17.131309 (+ 150us) write_op.cc:270] Start()
1211 20:55:17.131328 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510568170 usec, L: 0
1211 20:55:17.131530 (+ 202us) write_op.cc:270] Start()
1211 20:55:17.131553 (+ 23us) write_op.cc:276] Timestamp: P: 1765486510605737 usec, L: 0
1211 20:55:17.131775 (+ 222us) write_op.cc:270] Start()
1211 20:55:17.131805 (+ 30us) write_op.cc:276] Timestamp: P: 1765486510615936 usec, L: 0
1211 20:55:17.131971 (+ 166us) write_op.cc:270] Start()
1211 20:55:17.131991 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510651472 usec, L: 0
1211 20:55:17.132133 (+ 142us) write_op.cc:270] Start()
1211 20:55:17.132153 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510681249 usec, L: 0
1211 20:55:17.132305 (+ 152us) write_op.cc:270] Start()
1211 20:55:17.132324 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510688559 usec, L: 0
1211 20:55:17.132496 (+ 172us) write_op.cc:270] Start()
1211 20:55:17.132514 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510707354 usec, L: 0
1211 20:55:17.132647 (+ 133us) write_op.cc:270] Start()
1211 20:55:17.132665 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510720584 usec, L: 0
1211 20:55:17.132820 (+ 155us) write_op.cc:270] Start()
1211 20:55:17.132839 (+ 19us) write_op.cc:276] Timestamp: P: 1765486510746468 usec, L: 0
1211 20:55:17.132975 (+ 136us) write_op.cc:270] Start()
1211 20:55:17.132993 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510749759 usec, L: 0
1211 20:55:17.189107 (+ 56114us) write_op.cc:270] Start()
1211 20:55:17.189133 (+ 26us) write_op.cc:276] Timestamp: P: 1765486510750411 usec, L: 0
1211 20:55:17.191652 (+ 2519us) write_op.cc:270] Start()
1211 20:55:17.191682 (+ 30us) write_op.cc:276] Timestamp: P: 1765486510751043 usec, L: 0
1211 20:55:17.191867 (+ 185us) write_op.cc:270] Start()
1211 20:55:17.191889 (+ 22us) write_op.cc:276] Timestamp: P: 1765486510795671 usec, L: 0
1211 20:55:17.192055 (+ 166us) write_op.cc:270] Start()
1211 20:55:17.192076 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510812721 usec, L: 0
1211 20:55:17.192223 (+ 147us) write_op.cc:270] Start()
1211 20:55:17.192243 (+ 20us) write_op.cc:276] Timestamp: P: 1765486510818449 usec, L: 0
1211 20:55:17.192393 (+ 150us) write_op.cc:270] Start()
1211 20:55:17.192417 (+ 24us) write_op.cc:276] Timestamp: P: 1765486510824709 usec, L: 0
1211 20:55:17.192597 (+ 180us) write_op.cc:270] Start()
1211 20:55:17.192622 (+ 25us) write_op.cc:276] Timestamp: P: 1765486510838720 usec, L: 0
1211 20:55:17.194531 (+ 1909us) write_op.cc:270] Start()
1211 20:55:17.194556 (+ 25us) write_op.cc:276] Timestamp: P: 1765486510888356 usec, L: 0
1211 20:55:17.194713 (+ 157us) write_op.cc:270] Start()
1211 20:55:17.194734 (+ 21us) write_op.cc:276] Timestamp: P: 1765486510963744 usec, L: 0
1211 20:55:17.194873 (+ 139us) write_op.cc:270] Start()
1211 20:55:17.194891 (+ 18us) write_op.cc:276] Timestamp: P: 1765486510986671 usec, L: 0
1211 20:55:17.195018 (+ 127us) write_op.cc:270] Start()
1211 20:55:17.195036 (+ 18us) write_op.cc:276] Timestamp: P: 1765486511003519 usec, L: 0
1211 20:55:17.195173 (+ 137us) write_op.cc:270] Start()
1211 20:55:17.195194 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511015782 usec, L: 0
1211 20:55:17.195360 (+ 166us) write_op.cc:270] Start()
1211 20:55:17.195391 (+ 31us) write_op.cc:276] Timestamp: P: 1765486511054408 usec, L: 0
1211 20:55:17.195541 (+ 150us) write_op.cc:270] Start()
1211 20:55:17.195561 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511064149 usec, L: 0
1211 20:55:17.195713 (+ 152us) write_op.cc:270] Start()
1211 20:55:17.195732 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511091745 usec, L: 0
1211 20:55:17.214200 (+ 18468us) write_op.cc:270] Start()
1211 20:55:17.214225 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511113373 usec, L: 0
1211 20:55:17.214406 (+ 181us) write_op.cc:270] Start()
1211 20:55:17.214427 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511116658 usec, L: 0
1211 20:55:17.214627 (+ 200us) write_op.cc:270] Start()
1211 20:55:17.214649 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511120295 usec, L: 0
1211 20:55:17.364772 (+150123us) write_op.cc:270] Start()
1211 20:55:17.364798 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511152999 usec, L: 0
1211 20:55:17.369478 (+ 4680us) write_op.cc:270] Start()
1211 20:55:17.369501 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511211243 usec, L: 0
1211 20:55:17.378117 (+ 8616us) write_op.cc:270] Start()
1211 20:55:17.378155 (+ 38us) write_op.cc:276] Timestamp: P: 1765486511212021 usec, L: 0
1211 20:55:17.379505 (+ 1350us) write_op.cc:270] Start()
1211 20:55:17.379529 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511224907 usec, L: 0
1211 20:55:17.379691 (+ 162us) write_op.cc:270] Start()
1211 20:55:17.379713 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511232894 usec, L: 0
1211 20:55:17.458807 (+ 79094us) write_op.cc:270] Start()
1211 20:55:17.458830 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511278226 usec, L: 0
1211 20:55:17.465161 (+ 6331us) write_op.cc:270] Start()
1211 20:55:17.465197 (+ 36us) write_op.cc:276] Timestamp: P: 1765486511285698 usec, L: 0
1211 20:55:17.467774 (+ 2577us) write_op.cc:270] Start()
1211 20:55:17.467799 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511314181 usec, L: 0
1211 20:55:17.512024 (+ 44225us) write_op.cc:270] Start()
1211 20:55:17.512050 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511325345 usec, L: 0
1211 20:55:17.516079 (+ 4029us) write_op.cc:270] Start()
1211 20:55:17.518315 (+ 2236us) write_op.cc:276] Timestamp: P: 1765486511355370 usec, L: 0
1211 20:55:17.567495 (+ 49180us) write_op.cc:270] Start()
1211 20:55:17.567526 (+ 31us) write_op.cc:276] Timestamp: P: 1765486511366451 usec, L: 0
1211 20:55:17.574509 (+ 6983us) write_op.cc:270] Start()
1211 20:55:17.574556 (+ 47us) write_op.cc:276] Timestamp: P: 1765486511396797 usec, L: 0
1211 20:55:17.585732 (+ 11176us) write_op.cc:270] Start()
1211 20:55:17.585759 (+ 27us) write_op.cc:276] Timestamp: P: 1765486511399871 usec, L: 0
1211 20:55:17.599726 (+ 13967us) write_op.cc:270] Start()
1211 20:55:17.599757 (+ 31us) write_op.cc:276] Timestamp: P: 1765486511401202 usec, L: 0
1211 20:55:17.626056 (+ 26299us) write_op.cc:270] Start()
1211 20:55:17.626087 (+ 31us) write_op.cc:276] Timestamp: P: 1765486511483476 usec, L: 0
1211 20:55:17.626310 (+ 223us) write_op.cc:270] Start()
1211 20:55:17.626334 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511491638 usec, L: 0
1211 20:55:17.626499 (+ 165us) write_op.cc:270] Start()
1211 20:55:17.626521 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511499822 usec, L: 0
1211 20:55:17.636651 (+ 10130us) write_op.cc:270] Start()
1211 20:55:17.636688 (+ 37us) write_op.cc:276] Timestamp: P: 1765486511512510 usec, L: 0
1211 20:55:17.645340 (+ 8652us) write_op.cc:270] Start()
1211 20:55:17.645369 (+ 29us) write_op.cc:276] Timestamp: P: 1765486511523154 usec, L: 0
1211 20:55:17.647079 (+ 1710us) write_op.cc:270] Start()
1211 20:55:17.647109 (+ 30us) write_op.cc:276] Timestamp: P: 1765486511546580 usec, L: 0
1211 20:55:17.647304 (+ 195us) write_op.cc:270] Start()
1211 20:55:17.647327 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511573025 usec, L: 0
1211 20:55:17.654243 (+ 6916us) write_op.cc:270] Start()
1211 20:55:17.654268 (+ 25us) write_op.cc:276] Timestamp: P: 1765486511580705 usec, L: 0
1211 20:55:17.654444 (+ 176us) write_op.cc:270] Start()
1211 20:55:17.654463 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511581280 usec, L: 0
1211 20:55:17.654614 (+ 151us) write_op.cc:270] Start()
1211 20:55:17.654654 (+ 40us) write_op.cc:276] Timestamp: P: 1765486511620453 usec, L: 0
1211 20:55:17.659750 (+ 5096us) write_op.cc:270] Start()
1211 20:55:17.659776 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511644377 usec, L: 0
1211 20:55:17.659939 (+ 163us) write_op.cc:270] Start()
1211 20:55:17.659960 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511647214 usec, L: 0
1211 20:55:17.660178 (+ 218us) write_op.cc:270] Start()
1211 20:55:17.660198 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511662188 usec, L: 0
1211 20:55:17.660370 (+ 172us) write_op.cc:270] Start()
1211 20:55:17.660393 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511672564 usec, L: 0
1211 20:55:17.660534 (+ 141us) write_op.cc:270] Start()
1211 20:55:17.660555 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511715521 usec, L: 0
1211 20:55:17.660707 (+ 152us) write_op.cc:270] Start()
1211 20:55:17.660731 (+ 24us) write_op.cc:276] Timestamp: P: 1765486511738650 usec, L: 0
1211 20:55:17.660874 (+ 143us) write_op.cc:270] Start()
1211 20:55:17.660894 (+ 20us) write_op.cc:276] Timestamp: P: 1765486511758065 usec, L: 0
1211 20:55:17.661052 (+ 158us) write_op.cc:270] Start()
1211 20:55:17.661074 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511765388 usec, L: 0
1211 20:55:17.661241 (+ 167us) write_op.cc:270] Start()
1211 20:55:17.661262 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511774618 usec, L: 0
1211 20:55:17.661432 (+ 170us) write_op.cc:270] Start()
1211 20:55:17.661451 (+ 19us) write_op.cc:276] Timestamp: P: 1765486511803793 usec, L: 0
1211 20:55:17.661607 (+ 156us) write_op.cc:270] Start()
1211 20:55:17.661630 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511814981 usec, L: 0
1211 20:55:17.661802 (+ 172us) write_op.cc:270] Start()
1211 20:55:17.661823 (+ 21us) write_op.cc:276] Timestamp: P: 1765486511831107 usec, L: 0
1211 20:55:17.664414 (+ 2591us) write_op.cc:270] Start()
1211 20:55:17.664437 (+ 23us) write_op.cc:276] Timestamp: P: 1765486511848040 usec, L: 0
1211 20:55:17.666127 (+ 1690us) write_op.cc:270] Start()
1211 20:55:17.666149 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511859994 usec, L: 0
1211 20:55:17.671073 (+ 4924us) write_op.cc:270] Start()
1211 20:55:17.671099 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511887131 usec, L: 0
1211 20:55:17.677316 (+ 6217us) write_op.cc:270] Start()
1211 20:55:17.677343 (+ 27us) write_op.cc:276] Timestamp: P: 1765486511890358 usec, L: 0
1211 20:55:17.684137 (+ 6794us) write_op.cc:270] Start()
1211 20:55:17.684165 (+ 28us) write_op.cc:276] Timestamp: P: 1765486511930289 usec, L: 0
1211 20:55:17.688528 (+ 4363us) write_op.cc:270] Start()
1211 20:55:17.688572 (+ 44us) write_op.cc:276] Timestamp: P: 1765486511960159 usec, L: 0
1211 20:55:17.695347 (+ 6775us) write_op.cc:270] Start()
1211 20:55:17.695373 (+ 26us) write_op.cc:276] Timestamp: P: 1765486511979142 usec, L: 0
1211 20:55:17.698482 (+ 3109us) write_op.cc:270] Start()
1211 20:55:17.698504 (+ 22us) write_op.cc:276] Timestamp: P: 1765486511984991 usec, L: 0
1211 20:55:17.704781 (+ 6277us) write_op.cc:270] Start()
1211 20:55:17.704809 (+ 28us) write_op.cc:276] Timestamp: P: 1765486512003510 usec, L: 0
1211 20:55:17.720596 (+ 15787us) write_op.cc:270] Start()
1211 20:55:17.720622 (+ 26us) write_op.cc:276] Timestamp: P: 1765486512018405 usec, L: 0
1211 20:55:17.727768 (+ 7146us) write_op.cc:270] Start()
1211 20:55:17.727792 (+ 24us) write_op.cc:276] Timestamp: P: 1765486512049126 usec, L: 0
1211 20:55:17.737795 (+ 10003us) write_op.cc:270] Start()
1211 20:55:17.737820 (+ 25us) write_op.cc:276] Timestamp: P: 1765486512063024 usec, L: 0
1211 20:55:17.738021 (+ 201us) write_op.cc:270] Start()
1211 20:55:17.738045 (+ 24us) write_op.cc:276] Timestamp: P: 1765486512065830 usec, L: 0
1211 20:55:17.738220 (+ 175us) write_op.cc:270] Start()
1211 20:55:17.738244 (+ 24us) write_op.cc:276] Timestamp: P: 1765486512099941 usec, L: 0
1211 20:55:17.740057 (+ 1813us) write_op.cc:270] Start()
1211 20:55:17.740081 (+ 24us) write_op.cc:276] Timestamp: P: 1765486512107288 usec, L: 0
1211 20:55:17.740258 (+ 177us) write_op.cc:270] Start()
1211 20:55:17.740278 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512112438 usec, L: 0
1211 20:55:17.740466 (+ 188us) write_op.cc:270] Start()
1211 20:55:17.740488 (+ 22us) write_op.cc:276] Timestamp: P: 1765486512114359 usec, L: 0
1211 20:55:17.740661 (+ 173us) write_op.cc:270] Start()
1211 20:55:17.740680 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512160712 usec, L: 0
1211 20:55:17.740802 (+ 122us) write_op.cc:270] Start()
1211 20:55:17.740822 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512174413 usec, L: 0
1211 20:55:17.740954 (+ 132us) write_op.cc:270] Start()
1211 20:55:17.740976 (+ 22us) write_op.cc:276] Timestamp: P: 1765486512186261 usec, L: 0
1211 20:55:17.741121 (+ 145us) write_op.cc:270] Start()
1211 20:55:17.741141 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512187194 usec, L: 0
1211 20:55:17.741271 (+ 130us) write_op.cc:270] Start()
1211 20:55:17.741289 (+ 18us) write_op.cc:276] Timestamp: P: 1765486512217191 usec, L: 0
1211 20:55:17.741435 (+ 146us) write_op.cc:270] Start()
1211 20:55:17.741454 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512236876 usec, L: 0
1211 20:55:17.741590 (+ 136us) write_op.cc:270] Start()
1211 20:55:17.741608 (+ 18us) write_op.cc:276] Timestamp: P: 1765486512253938 usec, L: 0
1211 20:55:17.741731 (+ 123us) write_op.cc:270] Start()
1211 20:55:17.741748 (+ 17us) write_op.cc:276] Timestamp: P: 1765486512276872 usec, L: 0
1211 20:55:17.741892 (+ 144us) write_op.cc:270] Start()
1211 20:55:17.741911 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512293021 usec, L: 0
1211 20:55:17.742061 (+ 150us) write_op.cc:270] Start()
1211 20:55:17.742079 (+ 18us) write_op.cc:276] Timestamp: P: 1765486512306795 usec, L: 0
1211 20:55:17.742300 (+ 221us) write_op.cc:270] Start()
1211 20:55:17.742324 (+ 24us) write_op.cc:276] Timestamp: P: 1765486512327721 usec, L: 0
1211 20:55:17.742493 (+ 169us) write_op.cc:270] Start()
1211 20:55:17.742513 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512340339 usec, L: 0
1211 20:55:17.742665 (+ 152us) write_op.cc:270] Start()
1211 20:55:17.742685 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512355412 usec, L: 0
1211 20:55:17.742827 (+ 142us) write_op.cc:270] Start()
1211 20:55:17.742847 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512357665 usec, L: 0
1211 20:55:17.742985 (+ 138us) write_op.cc:270] Start()
1211 20:55:17.743003 (+ 18us) write_op.cc:276] Timestamp: P: 1765486512388258 usec, L: 0
1211 20:55:17.743209 (+ 206us) write_op.cc:270] Start()
1211 20:55:17.743233 (+ 24us) write_op.cc:276] Timestamp: P: 1765486512427534 usec, L: 0
1211 20:55:17.743499 (+ 266us) write_op.cc:270] Start()
1211 20:55:17.743528 (+ 29us) write_op.cc:276] Timestamp: P: 1765486512434284 usec, L: 0
1211 20:55:17.756085 (+ 12557us) write_op.cc:270] Start()
1211 20:55:17.756119 (+ 34us) write_op.cc:276] Timestamp: P: 1765486512462884 usec, L: 0
1211 20:55:17.759926 (+ 3807us) write_op.cc:270] Start()
1211 20:55:17.759956 (+ 30us) write_op.cc:276] Timestamp: P: 1765486512481726 usec, L: 0
1211 20:55:17.760130 (+ 174us) write_op.cc:270] Start()
1211 20:55:17.760152 (+ 22us) write_op.cc:276] Timestamp: P: 1765486512490238 usec, L: 0
1211 20:55:17.760700 (+ 548us) write_op.cc:270] Start()
1211 20:55:17.760719 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512495992 usec, L: 0
1211 20:55:17.767779 (+ 7060us) write_op.cc:270] Start()
1211 20:55:17.767802 (+ 23us) write_op.cc:276] Timestamp: P: 1765486512527329 usec, L: 0
1211 20:55:17.776369 (+ 8567us) write_op.cc:270] Start()
1211 20:55:17.776397 (+ 28us) write_op.cc:276] Timestamp: P: 1765486512536266 usec, L: 0
1211 20:55:17.776570 (+ 173us) write_op.cc:270] Start()
1211 20:55:17.776593 (+ 23us) write_op.cc:276] Timestamp: P: 1765486512557914 usec, L: 0
1211 20:55:17.776794 (+ 201us) write_op.cc:270] Start()
1211 20:55:17.776818 (+ 24us) write_op.cc:276] Timestamp: P: 1765486512567630 usec, L: 0
1211 20:55:17.777023 (+ 205us) write_op.cc:270] Start()
1211 20:55:17.777044 (+ 21us) write_op.cc:276] Timestamp: P: 1765486512577390 usec, L: 0
1211 20:55:17.777232 (+ 188us) write_op.cc:270] Start()
1211 20:55:17.777255 (+ 23us) write_op.cc:276] Timestamp: P: 1765486512597433 usec, L: 0
1211 20:55:17.793339 (+ 16084us) write_op.cc:270] Start()
1211 20:55:17.795153 (+ 1814us) write_op.cc:276] Timestamp: P: 1765486512625944 usec, L: 0
1211 20:55:17.804980 (+ 9827us) write_op.cc:270] Start()
1211 20:55:17.805005 (+ 25us) write_op.cc:276] Timestamp: P: 1765486512626643 usec, L: 0
1211 20:55:17.805185 (+ 180us) write_op.cc:270] Start()
1211 20:55:17.805208 (+ 23us) write_op.cc:276] Timestamp: P: 1765486512639580 usec, L: 0
1211 20:55:17.805361 (+ 153us) write_op.cc:270] Start()
1211 20:55:17.805383 (+ 22us) write_op.cc:276] Timestamp: P: 1765486512659090 usec, L: 0
1211 20:55:17.805532 (+ 149us) write_op.cc:270] Start()
1211 20:55:17.805551 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512686570 usec, L: 0
1211 20:55:17.805717 (+ 166us) write_op.cc:270] Start()
1211 20:55:17.805736 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512696254 usec, L: 0
1211 20:55:17.805892 (+ 156us) write_op.cc:270] Start()
1211 20:55:17.805909 (+ 17us) write_op.cc:276] Timestamp: P: 1765486512702797 usec, L: 0
1211 20:55:17.806058 (+ 149us) write_op.cc:270] Start()
1211 20:55:17.806078 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512707014 usec, L: 0
1211 20:55:17.806226 (+ 148us) write_op.cc:270] Start()
1211 20:55:17.806245 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512727939 usec, L: 0
1211 20:55:17.806381 (+ 136us) write_op.cc:270] Start()
1211 20:55:17.806400 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512763307 usec, L: 0
1211 20:55:17.806543 (+ 143us) write_op.cc:270] Start()
1211 20:55:17.806564 (+ 21us) write_op.cc:276] Timestamp: P: 1765486512768055 usec, L: 0
1211 20:55:17.806718 (+ 154us) write_op.cc:270] Start()
1211 20:55:17.806737 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512795013 usec, L: 0
1211 20:55:17.806860 (+ 123us) write_op.cc:270] Start()
1211 20:55:17.806880 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512815748 usec, L: 0
1211 20:55:17.807086 (+ 206us) write_op.cc:270] Start()
1211 20:55:17.807108 (+ 22us) write_op.cc:276] Timestamp: P: 1765486512826968 usec, L: 0
1211 20:55:17.807264 (+ 156us) write_op.cc:270] Start()
1211 20:55:17.807284 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512845707 usec, L: 0
1211 20:55:17.807435 (+ 151us) write_op.cc:270] Start()
1211 20:55:17.807454 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512865730 usec, L: 0
1211 20:55:17.807599 (+ 145us) write_op.cc:270] Start()
1211 20:55:17.807619 (+ 20us) write_op.cc:276] Timestamp: P: 1765486512879204 usec, L: 0
1211 20:55:17.807786 (+ 167us) write_op.cc:270] Start()
1211 20:55:17.807807 (+ 21us) write_op.cc:276] Timestamp: P: 1765486512918438 usec, L: 0
1211 20:55:17.807960 (+ 153us) write_op.cc:270] Start()
1211 20:55:17.807979 (+ 19us) write_op.cc:276] Timestamp: P: 1765486512924190 usec, L: 0
1211 20:55:17.808110 (+ 131us) write_op.cc:270] Start()
1211 20:55:17.808140 (+ 30us) write_op.cc:276] Timestamp: P: 1765486512932478 usec, L: 0
1211 20:55:17.808309 (+ 169us) write_op.cc:270] Start()
1211 20:55:17.808326 (+ 17us) write_op.cc:276] Timestamp: P: 1765486512956252 usec, L: 0
1211 20:55:17.825061 (+ 16735us) write_op.cc:270] Start()
1211 20:55:17.825089 (+ 28us) write_op.cc:276] Timestamp: P: 1765486512987170 usec, L: 0
1211 20:55:17.825241 (+ 152us) write_op.cc:270] Start()
1211 20:55:17.825258 (+ 17us) write_op.cc:276] Timestamp: P: 1765486512997426 usec, L: 0
1211 20:55:17.825404 (+ 146us) write_op.cc:270] Start()
1211 20:55:17.825422 (+ 18us) write_op.cc:276] Timestamp: P: 1765486513019917 usec, L: 0
1211 20:55:17.825577 (+ 155us) write_op.cc:270] Start()
1211 20:55:17.825600 (+ 23us) write_op.cc:276] Timestamp: P: 1765486513028577 usec, L: 0
1211 20:55:17.825733 (+ 133us) write_op.cc:270] Start()
1211 20:55:17.825748 (+ 15us) write_op.cc:276] Timestamp: P: 1765486513045408 usec, L: 0
1211 20:55:17.825891 (+ 143us) write_op.cc:270] Start()
1211 20:55:17.825905 (+ 14us) write_op.cc:276] Timestamp: P: 1765486513052092 usec, L: 0
1211 20:55:17.826030 (+ 125us) write_op.cc:270] Start()
1211 20:55:17.826046 (+ 16us) write_op.cc:276] Timestamp: P: 1765486513068548 usec, L: 0
1211 20:55:17.826181 (+ 135us) write_op.cc:270] Start()
1211 20:55:17.826204 (+ 23us) write_op.cc:276] Timestamp: P: 1765486513086154 usec, L: 0
1211 20:55:17.826354 (+ 150us) write_op.cc:270] Start()
1211 20:55:17.826383 (+ 29us) write_op.cc:276] Timestamp: P: 1765486513100713 usec, L: 0
1211 20:55:17.826551 (+ 168us) write_op.cc:270] Start()
1211 20:55:17.826571 (+ 20us) write_op.cc:276] Timestamp: P: 1765486513136489 usec, L: 0
1211 20:55:17.826774 (+ 203us) write_op.cc:270] Start()
1211 20:55:17.826797 (+ 23us) write_op.cc:276] Timestamp: P: 1765486513140086 usec, L: 0
1211 20:55:17.827007 (+ 210us) write_op.cc:270] Start()
1211 20:55:17.827027 (+ 20us) write_op.cc:276] Timestamp: P: 1765486513151392 usec, L: 0
1211 20:55:17.827188 (+ 161us) write_op.cc:270] Start()
1211 20:55:17.827207 (+ 19us) write_op.cc:276] Timestamp: P: 1765486513171470 usec, L: 0
1211 20:55:17.827396 (+ 189us) write_op.cc:270] Start()
1211 20:55:17.827416 (+ 20us) write_op.cc:276] Timestamp: P: 1765486513178251 usec, L: 0
1211 20:55:17.827562 (+ 146us) write_op.cc:270] Start()
1211 20:55:17.827579 (+ 17us) write_op.cc:276] Timestamp: P: 1765486513179170 usec, L: 0
1211 20:55:17.827767 (+ 188us) write_op.cc:270] Start()
1211 20:55:17.827787 (+ 20us) write_op.cc:276] Timestamp: P: 1765486513208866 usec, L: 0
1211 20:55:17.827946 (+ 159us) write_op.cc:270] Start()
1211 20:55:17.827965 (+ 19us) write_op.cc:276] Timestamp: P: 1765486513243069 usec, L: 0
1211 20:55:17.828101 (+ 136us) write_op.cc:270] Start()
1211 20:55:17.828124 (+ 23us) write_op.cc:276] Timestamp: P: 1765486513258420 usec, L: 0
1211 20:55:17.828343 (+ 219us) write_op.cc:270] Start()
1211 20:55:17.828374 (+ 31us) write_op.cc:276] Timestamp: P: 1765486513273444 usec, L: 0
1211 20:55:17.828518 (+ 144us) write_op.cc:270] Start()
1211 20:55:17.828540 (+ 22us) write_op.cc:276] Timestamp: P: 1765486513287198 usec, L: 0
1211 20:55:17.828702 (+ 162us) write_op.cc:270] Start()
1211 20:55:17.828739 (+ 37us) write_op.cc:276] Timestamp: P: 1765486513303094 usec, L: 0
1211 20:55:17.828902 (+ 163us) write_op.cc:270] Start()
1211 20:55:17.828921 (+ 19us) write_op.cc:276] Timestamp: P: 1765486513321569 usec, L: 0
1211 20:55:17.829046 (+ 125us) write_op.cc:270] Start()
1211 20:55:17.829063 (+ 17us) write_op.cc:276] Timestamp: P: 1765486513352972 usec, L: 0
1211 20:55:17.829194 (+ 131us) write_op.cc:270] Start()
1211 20:55:17.829213 (+ 19us) write_op.cc:276] Timestamp: P: 1765486513353597 usec, L: 0
1211 20:55:17.829353 (+ 140us) write_op.cc:270] Start()
1211 20:55:17.829383 (+ 30us) write_op.cc:276] Timestamp: P: 1765486513365862 usec, L: 0
1211 20:55:17.829549 (+ 166us) write_op.cc:270] Start()
1211 20:55:17.829571 (+ 22us) write_op.cc:276] Timestamp: P: 1765486513371773 usec, L: 0
1211 20:55:17.829756 (+ 185us) write_op.cc:270] Start()
1211 20:55:17.829775 (+ 19us) write_op.cc:276] Timestamp: P: 1765486513427995 usec, L: 0
1211 20:55:17.873271 (+ 43496us) write_op.cc:270] Start()
1211 20:55:17.873305 (+ 34us) write_op.cc:276] Timestamp: P: 1765486513448921 usec, L: 0
1211 20:55:17.873591 (+ 286us) write_op.cc:270] Start()
1211 20:55:17.873616 (+ 25us) write_op.cc:276] Timestamp: P: 1765486513475456 usec, L: 0
1211 20:55:17.873787 (+ 171us) write_op.cc:270] Start()
1211 20:55:17.873810 (+ 23us) write_op.cc:276] Timestamp: P: 1765486513476377 usec, L: 0
1211 20:55:17.873963 (+ 153us) write_op.cc:270] Start()
1211 20:55:17.873984 (+ 21us) write_op.cc:276] Timestamp: P: 1765486513495753 usec, L: 0
1211 20:55:17.877426 (+ 3442us) write_op.cc:270] Start()
1211 20:55:17.877451 (+ 25us) write_op.cc:276] Timestamp: P: 1765486513501263 usec, L: 0
1211 20:55:17.902086 (+ 24635us) write_op.cc:270] Start()
1211 20:55:17.902108 (+ 22us) write_op.cc:276] Timestamp: P: 1765486513549546 usec, L: 0
1211 20:55:17.909056 (+ 6948us) write_op.cc:270] Start()
1211 20:55:17.909084 (+ 28us) write_op.cc:276] Timestamp: P: 1765486513553289 usec, L: 0
1211 20:55:17.909251 (+ 167us) write_op.cc:270] Start()
1211 20:55:17.909271 (+ 20us) write_op.cc:276] Timestamp: P: 1765486513559412 usec, L: 0
1211 20:55:17.911168 (+ 1897us) write_op.cc:270] Start()
1211 20:55:17.911193 (+ 25us) write_op.cc:276] Timestamp: P: 1765486513567451 usec, L: 0
1211 20:55:17.917344 (+ 6151us) write_op.cc:270] Start()
1211 20:55:17.917372 (+ 28us) write_op.cc:276] Timestamp: P: 1765486513595507 usec, L: 0
1211 20:55:17.917583 (+ 211us) write_op.cc:270] Start()
1211 20:55:17.917606 (+ 23us) write_op.cc:276] Timestamp: P: 1765486513610531 usec, L: 0
1211 20:55:17.918611 (+ 1005us) write_op.cc:270] Start()
1211 20:55:17.918636 (+ 25us) write_op.cc:276] Timestamp: P: 1765486513627566 usec, L: 0
1211 20:55:17.918803 (+ 167us) write_op.cc:270] Start()
1211 20:55:17.918822 (+ 19us) write_op.cc:276] Timestamp: P: 1765486513630308 usec, L: 0
1211 20:55:17.918981 (+ 159us) write_op.cc:270] Start()
1211 20:55:17.919003 (+ 22us) write_op.cc:276] Timestamp: P: 1765486513643602 usec, L: 0
1211 20:55:17.919203 (+ 200us) write_op.cc:270] Start()
1211 20:55:17.919227 (+ 24us) write_op.cc:276] Timestamp: P: 1765486513655659 usec, L: 0
1211 20:55:17.942196 (+ 22969us) write_op.cc:270] Start()
1211 20:55:17.942225 (+ 29us) write_op.cc:276] Timestamp: P: 1765486513674083 usec, L: 0
1211 20:55:17.942410 (+ 185us) write_op.cc:270] Start()
1211 20:55:17.942432 (+ 22us) write_op.cc:276] Timestamp: P: 1765486513717970 usec, L: 0
1211 20:55:17.942568 (+ 136us) write_op.cc:270] Start()
1211 20:55:17.942588 (+ 20us) write_op.cc:276] Timestamp: P: 1765486513719835 usec, L: 0
1211 20:55:17.942723 (+ 135us) write_op.cc:270] Start()
1211 20:55:17.942742 (+ 19us) write_op.cc:276] Timestamp: P: 176548
I20251211 20:55:20.043062 7078 raft_consensus.cc:493] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:20.043313 7078 raft_consensus.cc:515] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:20.071130 7078 leader_election.cc:290] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:20.092660 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1489 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:20.095623 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "ce1e22ac4b814a508b20688b15d0c959" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1489 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:20.151811 6872 leader_election.cc:304] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:20.153239 7078 raft_consensus.cc:2749] T ce1e22ac4b814a508b20688b15d0c959 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:20.549743 7079 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:20.549978 7079 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:20.572957 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1490 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:20.578433 7079 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:20.591462 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "0b788bd87c9a4ca8a88d144e2af0c6c9" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1490 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:20.593479 6872 leader_election.cc:304] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:20.756223 7079 raft_consensus.cc:2749] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:21.899565 6943 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.5.211.194:47609 (request call id 19) took 6324 ms. Trace:
I20251211 20:55:21.900295 6901 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.ListTablets from 127.0.0.1:42702 (request call id 14) took 2507 ms. Trace:
I20251211 20:55:21.900409 6901 rpcz_store.cc:276] 1211 20:55:19.392333 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:19.393144 (+ 811us) service_pool.cc:225] Handling call
1211 20:55:21.899511 (+2506367us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x61600001a854. stack: 00007fdcd3329ee6 00007fdcd3329ca0 00007fdcd23053ad 0000000000984f94 0000000000984f5d 000000000098492a 00007fdcdf1c32c0 00007fdce3073b2f 00007fdcdd498c36 00007fdcdd498965 00007fdcda2c956e 00007fdcda2c844e 00007fdcda2ce685 00007fdcda2cefaf 00007fdcda2cedd8 0000000000a8970c
1211 20:55:21.900237 (+ 726us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5758235008}
I20251211 20:55:21.900714 7015 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:21.900915 7015 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:21.902688 6278 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1276 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:21.902034 7015 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:21.915611 6407 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1276 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:21.928488 6872 leader_election.cc:304] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:21.929618 7015 raft_consensus.cc:2749] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:22.378244 7015 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:22.378616 7015 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:22.379247 7015 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:22.389555 6408 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1276 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:22.389570 6276 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1276 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:22.396512 6872 leader_election.cc:304] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:22.397815 7015 raft_consensus.cc:2749] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:22.705768 7015 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:22.705971 7015 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:22.706760 7015 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:22.707072 6275 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1276 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:22.743454 6406 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1276 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:22.764061 6872 leader_election.cc:304] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:22.764438 7015 raft_consensus.cc:2749] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:21.899803 6943 rpcz_store.cc:276] 1211 20:55:15.574914 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:15.575016 (+ 102us) service_pool.cc:225] Handling call
1211 20:55:16.216958 (+641942us) raft_consensus.cc:1471] Updating replica for 1116 ops
1211 20:55:17.083386 (+866428us) raft_consensus.cc:1534] Early marking committed up to index 160
1211 20:55:17.083471 (+ 85us) raft_consensus.cc:1539] Triggering prepare for 1116 ops
1211 20:55:17.083579 (+ 108us) write_op.cc:270] Start()
1211 20:55:17.083606 (+ 27us) write_op.cc:276] Timestamp: P: 1765486495504113 usec, L: 0
1211 20:55:17.106448 (+ 22842us) write_op.cc:270] Start()
1211 20:55:17.106477 (+ 29us) write_op.cc:276] Timestamp: P: 1765486495538081 usec, L: 0
1211 20:55:17.106646 (+ 169us) write_op.cc:270] Start()
1211 20:55:17.106667 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495543708 usec, L: 0
1211 20:55:17.106793 (+ 126us) write_op.cc:270] Start()
1211 20:55:17.106810 (+ 17us) write_op.cc:276] Timestamp: P: 1765486495559341 usec, L: 0
1211 20:55:17.106942 (+ 132us) write_op.cc:270] Start()
1211 20:55:17.106961 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495567114 usec, L: 0
1211 20:55:17.124672 (+ 17711us) write_op.cc:270] Start()
1211 20:55:17.124703 (+ 31us) write_op.cc:276] Timestamp: P: 1765486495577908 usec, L: 0
1211 20:55:17.124877 (+ 174us) write_op.cc:270] Start()
1211 20:55:17.124921 (+ 44us) write_op.cc:276] Timestamp: P: 1765486495609634 usec, L: 0
1211 20:55:17.125100 (+ 179us) write_op.cc:270] Start()
1211 20:55:17.125120 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495611002 usec, L: 0
1211 20:55:17.125280 (+ 160us) write_op.cc:270] Start()
1211 20:55:17.125299 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495623950 usec, L: 0
1211 20:55:17.125462 (+ 163us) write_op.cc:270] Start()
1211 20:55:17.125482 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495643328 usec, L: 0
1211 20:55:17.125622 (+ 140us) write_op.cc:270] Start()
1211 20:55:17.125639 (+ 17us) write_op.cc:276] Timestamp: P: 1765486495651627 usec, L: 0
1211 20:55:17.174106 (+ 48467us) write_op.cc:270] Start()
1211 20:55:17.174135 (+ 29us) write_op.cc:276] Timestamp: P: 1765486495676744 usec, L: 0
1211 20:55:17.174342 (+ 207us) write_op.cc:270] Start()
1211 20:55:17.174363 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495693838 usec, L: 0
1211 20:55:17.174528 (+ 165us) write_op.cc:270] Start()
1211 20:55:17.174550 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495694499 usec, L: 0
1211 20:55:17.174692 (+ 142us) write_op.cc:270] Start()
1211 20:55:17.174712 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495721686 usec, L: 0
1211 20:55:17.174856 (+ 144us) write_op.cc:270] Start()
1211 20:55:17.174883 (+ 27us) write_op.cc:276] Timestamp: P: 1765486495727484 usec, L: 0
1211 20:55:17.175060 (+ 177us) write_op.cc:270] Start()
1211 20:55:17.175079 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495745859 usec, L: 0
1211 20:55:17.175215 (+ 136us) write_op.cc:270] Start()
1211 20:55:17.175236 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495760677 usec, L: 0
1211 20:55:17.175380 (+ 144us) write_op.cc:270] Start()
1211 20:55:17.175402 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495768811 usec, L: 0
1211 20:55:17.175576 (+ 174us) write_op.cc:270] Start()
1211 20:55:17.175597 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495799596 usec, L: 0
1211 20:55:17.175834 (+ 237us) write_op.cc:270] Start()
1211 20:55:17.175856 (+ 22us) write_op.cc:276] Timestamp: P: 1765486495803081 usec, L: 0
1211 20:55:17.176027 (+ 171us) write_op.cc:270] Start()
1211 20:55:17.176048 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495836618 usec, L: 0
1211 20:55:17.206213 (+ 30165us) write_op.cc:270] Start()
1211 20:55:17.206237 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495859928 usec, L: 0
1211 20:55:17.212236 (+ 5999us) write_op.cc:270] Start()
1211 20:55:17.212265 (+ 29us) write_op.cc:276] Timestamp: P: 1765486495882606 usec, L: 0
1211 20:55:17.223786 (+ 11521us) write_op.cc:270] Start()
1211 20:55:17.223826 (+ 40us) write_op.cc:276] Timestamp: P: 1765486495892800 usec, L: 0
1211 20:55:17.224032 (+ 206us) write_op.cc:270] Start()
1211 20:55:17.224056 (+ 24us) write_op.cc:276] Timestamp: P: 1765486495897418 usec, L: 0
1211 20:55:17.224252 (+ 196us) write_op.cc:270] Start()
1211 20:55:17.224272 (+ 20us) write_op.cc:276] Timestamp: P: 1765486495910434 usec, L: 0
1211 20:55:17.224415 (+ 143us) write_op.cc:270] Start()
1211 20:55:17.224434 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495932350 usec, L: 0
1211 20:55:17.224580 (+ 146us) write_op.cc:270] Start()
1211 20:55:17.224598 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495954922 usec, L: 0
1211 20:55:17.224742 (+ 144us) write_op.cc:270] Start()
1211 20:55:17.224761 (+ 19us) write_op.cc:276] Timestamp: P: 1765486495958717 usec, L: 0
1211 20:55:17.224913 (+ 152us) write_op.cc:270] Start()
1211 20:55:17.224931 (+ 18us) write_op.cc:276] Timestamp: P: 1765486495961374 usec, L: 0
1211 20:55:17.225065 (+ 134us) write_op.cc:270] Start()
1211 20:55:17.225086 (+ 21us) write_op.cc:276] Timestamp: P: 1765486495969387 usec, L: 0
1211 20:55:17.225234 (+ 148us) write_op.cc:270] Start()
1211 20:55:17.225251 (+ 17us) write_op.cc:276] Timestamp: P: 1765486495996192 usec, L: 0
1211 20:55:17.225389 (+ 138us) write_op.cc:270] Start()
1211 20:55:17.225408 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496010072 usec, L: 0
1211 20:55:17.225542 (+ 134us) write_op.cc:270] Start()
1211 20:55:17.225559 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496020320 usec, L: 0
1211 20:55:17.225691 (+ 132us) write_op.cc:270] Start()
1211 20:55:17.225729 (+ 38us) write_op.cc:276] Timestamp: P: 1765486496031144 usec, L: 0
1211 20:55:17.362832 (+137103us) write_op.cc:270] Start()
1211 20:55:17.362871 (+ 39us) write_op.cc:276] Timestamp: P: 1765486496057833 usec, L: 0
1211 20:55:17.363117 (+ 246us) write_op.cc:270] Start()
1211 20:55:17.363158 (+ 41us) write_op.cc:276] Timestamp: P: 1765486496058574 usec, L: 0
1211 20:55:17.363377 (+ 219us) write_op.cc:270] Start()
1211 20:55:17.363401 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496075074 usec, L: 0
1211 20:55:17.363561 (+ 160us) write_op.cc:270] Start()
1211 20:55:17.363582 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496084618 usec, L: 0
1211 20:55:17.363790 (+ 208us) write_op.cc:270] Start()
1211 20:55:17.363811 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496107189 usec, L: 0
1211 20:55:17.364014 (+ 203us) write_op.cc:270] Start()
1211 20:55:17.364034 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496116647 usec, L: 0
1211 20:55:17.364184 (+ 150us) write_op.cc:270] Start()
1211 20:55:17.364203 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496119375 usec, L: 0
1211 20:55:17.364375 (+ 172us) write_op.cc:270] Start()
1211 20:55:17.364397 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496141599 usec, L: 0
1211 20:55:17.364573 (+ 176us) write_op.cc:270] Start()
1211 20:55:17.364595 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496144458 usec, L: 0
1211 20:55:17.364796 (+ 201us) write_op.cc:270] Start()
1211 20:55:17.364812 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496156023 usec, L: 0
1211 20:55:17.364961 (+ 149us) write_op.cc:270] Start()
1211 20:55:17.364981 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496156674 usec, L: 0
1211 20:55:17.365184 (+ 203us) write_op.cc:270] Start()
1211 20:55:17.365205 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496189926 usec, L: 0
1211 20:55:17.365323 (+ 118us) write_op.cc:270] Start()
1211 20:55:17.365339 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496191125 usec, L: 0
1211 20:55:17.365478 (+ 139us) write_op.cc:270] Start()
1211 20:55:17.365496 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496196163 usec, L: 0
1211 20:55:17.365614 (+ 118us) write_op.cc:270] Start()
1211 20:55:17.365639 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496212194 usec, L: 0
1211 20:55:17.365758 (+ 119us) write_op.cc:270] Start()
1211 20:55:17.365774 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496239157 usec, L: 0
1211 20:55:17.365897 (+ 123us) write_op.cc:270] Start()
1211 20:55:17.365913 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496247926 usec, L: 0
1211 20:55:17.366184 (+ 271us) write_op.cc:270] Start()
1211 20:55:17.366204 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496264086 usec, L: 0
1211 20:55:17.366324 (+ 120us) write_op.cc:270] Start()
1211 20:55:17.366340 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496285725 usec, L: 0
1211 20:55:17.366469 (+ 129us) write_op.cc:270] Start()
1211 20:55:17.366485 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496287205 usec, L: 0
1211 20:55:17.366606 (+ 121us) write_op.cc:270] Start()
1211 20:55:17.366623 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496293006 usec, L: 0
1211 20:55:17.366745 (+ 122us) write_op.cc:270] Start()
1211 20:55:17.366761 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496316313 usec, L: 0
1211 20:55:17.368358 (+ 1597us) write_op.cc:270] Start()
1211 20:55:17.368383 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496335590 usec, L: 0
1211 20:55:17.368532 (+ 149us) write_op.cc:270] Start()
1211 20:55:17.369340 (+ 808us) write_op.cc:276] Timestamp: P: 1765486496336164 usec, L: 0
1211 20:55:17.370679 (+ 1339us) write_op.cc:270] Start()
1211 20:55:17.370702 (+ 23us) write_op.cc:276] Timestamp: P: 1765486496356230 usec, L: 0
1211 20:55:17.370857 (+ 155us) write_op.cc:270] Start()
1211 20:55:17.370877 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496379340 usec, L: 0
1211 20:55:17.371058 (+ 181us) write_op.cc:270] Start()
1211 20:55:17.371250 (+ 192us) write_op.cc:276] Timestamp: P: 1765486496392005 usec, L: 0
1211 20:55:17.371456 (+ 206us) write_op.cc:270] Start()
1211 20:55:17.371480 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496410353 usec, L: 0
1211 20:55:17.373144 (+ 1664us) write_op.cc:270] Start()
1211 20:55:17.373168 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496412078 usec, L: 0
1211 20:55:17.448979 (+ 75811us) write_op.cc:270] Start()
1211 20:55:17.449006 (+ 27us) write_op.cc:276] Timestamp: P: 1765486496417105 usec, L: 0
1211 20:55:17.457248 (+ 8242us) write_op.cc:270] Start()
1211 20:55:17.457277 (+ 29us) write_op.cc:276] Timestamp: P: 1765486496437507 usec, L: 0
1211 20:55:17.458605 (+ 1328us) write_op.cc:270] Start()
1211 20:55:17.458633 (+ 28us) write_op.cc:276] Timestamp: P: 1765486496448327 usec, L: 0
1211 20:55:17.460908 (+ 2275us) write_op.cc:270] Start()
1211 20:55:17.460930 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496470647 usec, L: 0
1211 20:55:17.463262 (+ 2332us) write_op.cc:270] Start()
1211 20:55:17.463286 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496476039 usec, L: 0
1211 20:55:17.466584 (+ 3298us) write_op.cc:270] Start()
1211 20:55:17.466606 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496483571 usec, L: 0
1211 20:55:17.509767 (+ 43161us) write_op.cc:270] Start()
1211 20:55:17.509797 (+ 30us) write_op.cc:276] Timestamp: P: 1765486496487280 usec, L: 0
1211 20:55:17.510061 (+ 264us) write_op.cc:270] Start()
1211 20:55:17.510086 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496508431 usec, L: 0
1211 20:55:17.510279 (+ 193us) write_op.cc:270] Start()
1211 20:55:17.510299 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496526429 usec, L: 0
1211 20:55:17.510486 (+ 187us) write_op.cc:270] Start()
1211 20:55:17.510508 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496531317 usec, L: 0
1211 20:55:17.510653 (+ 145us) write_op.cc:270] Start()
1211 20:55:17.510674 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496538349 usec, L: 0
1211 20:55:17.510813 (+ 139us) write_op.cc:270] Start()
1211 20:55:17.510833 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496558796 usec, L: 0
1211 20:55:17.510998 (+ 165us) write_op.cc:270] Start()
1211 20:55:17.511038 (+ 40us) write_op.cc:276] Timestamp: P: 1765486496575244 usec, L: 0
1211 20:55:17.511214 (+ 176us) write_op.cc:270] Start()
1211 20:55:17.511235 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496583256 usec, L: 0
1211 20:55:17.577607 (+ 66372us) write_op.cc:270] Start()
1211 20:55:17.577631 (+ 24us) write_op.cc:276] Timestamp: P: 1765486496583805 usec, L: 0
1211 20:55:17.587200 (+ 9569us) write_op.cc:270] Start()
1211 20:55:17.587228 (+ 28us) write_op.cc:276] Timestamp: P: 1765486496604748 usec, L: 0
1211 20:55:17.587407 (+ 179us) write_op.cc:270] Start()
1211 20:55:17.587429 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496619660 usec, L: 0
1211 20:55:17.598195 (+ 10766us) write_op.cc:270] Start()
1211 20:55:17.598227 (+ 32us) write_op.cc:276] Timestamp: P: 1765486496627647 usec, L: 0
1211 20:55:17.624065 (+ 25838us) write_op.cc:270] Start()
1211 20:55:17.624098 (+ 33us) write_op.cc:276] Timestamp: P: 1765486496640935 usec, L: 0
1211 20:55:17.624302 (+ 204us) write_op.cc:270] Start()
1211 20:55:17.624327 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496650047 usec, L: 0
1211 20:55:17.624591 (+ 264us) write_op.cc:270] Start()
1211 20:55:17.624616 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496676088 usec, L: 0
1211 20:55:17.624776 (+ 160us) write_op.cc:270] Start()
1211 20:55:17.624797 (+ 21us) write_op.cc:276] Timestamp: P: 1765486496688244 usec, L: 0
1211 20:55:17.624928 (+ 131us) write_op.cc:270] Start()
1211 20:55:17.624945 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496691504 usec, L: 0
1211 20:55:17.625147 (+ 202us) write_op.cc:270] Start()
1211 20:55:17.625165 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496695514 usec, L: 0
1211 20:55:17.625300 (+ 135us) write_op.cc:270] Start()
1211 20:55:17.625319 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496709930 usec, L: 0
1211 20:55:17.625466 (+ 147us) write_op.cc:270] Start()
1211 20:55:17.625484 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496743189 usec, L: 0
1211 20:55:17.625625 (+ 141us) write_op.cc:270] Start()
1211 20:55:17.625644 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496752838 usec, L: 0
1211 20:55:17.625781 (+ 137us) write_op.cc:270] Start()
1211 20:55:17.625797 (+ 16us) write_op.cc:276] Timestamp: P: 1765486496754368 usec, L: 0
1211 20:55:17.625950 (+ 153us) write_op.cc:270] Start()
1211 20:55:17.625967 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496783086 usec, L: 0
1211 20:55:17.626161 (+ 194us) write_op.cc:270] Start()
1211 20:55:17.626181 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496790487 usec, L: 0
1211 20:55:17.626340 (+ 159us) write_op.cc:270] Start()
1211 20:55:17.626358 (+ 18us) write_op.cc:276] Timestamp: P: 1765486496813428 usec, L: 0
1211 20:55:17.639863 (+ 13505us) write_op.cc:270] Start()
1211 20:55:17.639888 (+ 25us) write_op.cc:276] Timestamp: P: 1765486496828325 usec, L: 0
1211 20:55:17.640039 (+ 151us) write_op.cc:270] Start()
1211 20:55:17.640059 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496829298 usec, L: 0
1211 20:55:17.640207 (+ 148us) write_op.cc:270] Start()
1211 20:55:17.640224 (+ 17us) write_op.cc:276] Timestamp: P: 1765486496854616 usec, L: 0
1211 20:55:17.640333 (+ 109us) write_op.cc:270] Start()
1211 20:55:17.640347 (+ 14us) write_op.cc:276] Timestamp: P: 1765486496866456 usec, L: 0
1211 20:55:17.640450 (+ 103us) write_op.cc:270] Start()
1211 20:55:17.640465 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496873515 usec, L: 0
1211 20:55:17.640605 (+ 140us) write_op.cc:270] Start()
1211 20:55:17.640637 (+ 32us) write_op.cc:276] Timestamp: P: 1765486496900325 usec, L: 0
1211 20:55:17.640780 (+ 143us) write_op.cc:270] Start()
1211 20:55:17.640795 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496920543 usec, L: 0
1211 20:55:17.640898 (+ 103us) write_op.cc:270] Start()
1211 20:55:17.640939 (+ 41us) write_op.cc:276] Timestamp: P: 1765486496925250 usec, L: 0
1211 20:55:17.641077 (+ 138us) write_op.cc:270] Start()
1211 20:55:17.641097 (+ 20us) write_op.cc:276] Timestamp: P: 1765486496940524 usec, L: 0
1211 20:55:17.641231 (+ 134us) write_op.cc:270] Start()
1211 20:55:17.641262 (+ 31us) write_op.cc:276] Timestamp: P: 1765486496958903 usec, L: 0
1211 20:55:17.641431 (+ 169us) write_op.cc:270] Start()
1211 20:55:17.641453 (+ 22us) write_op.cc:276] Timestamp: P: 1765486496976567 usec, L: 0
1211 20:55:17.641597 (+ 144us) write_op.cc:270] Start()
1211 20:55:17.641612 (+ 15us) write_op.cc:276] Timestamp: P: 1765486496993030 usec, L: 0
1211 20:55:17.641780 (+ 168us) write_op.cc:270] Start()
1211 20:55:17.641799 (+ 19us) write_op.cc:276] Timestamp: P: 1765486496999508 usec, L: 0
1211 20:55:17.642758 (+ 959us) write_op.cc:270] Start()
1211 20:55:17.642784 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497002524 usec, L: 0
1211 20:55:17.642933 (+ 149us) write_op.cc:270] Start()
1211 20:55:17.642952 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497025862 usec, L: 0
1211 20:55:17.643194 (+ 242us) write_op.cc:270] Start()
1211 20:55:17.644006 (+ 812us) write_op.cc:276] Timestamp: P: 1765486497040737 usec, L: 0
1211 20:55:17.644202 (+ 196us) write_op.cc:270] Start()
1211 20:55:17.644235 (+ 33us) write_op.cc:276] Timestamp: P: 1765486497061037 usec, L: 0
1211 20:55:17.644430 (+ 195us) write_op.cc:270] Start()
1211 20:55:17.644450 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497077093 usec, L: 0
1211 20:55:17.649528 (+ 5078us) write_op.cc:270] Start()
1211 20:55:17.649557 (+ 29us) write_op.cc:276] Timestamp: P: 1765486497083786 usec, L: 0
1211 20:55:17.665406 (+ 15849us) write_op.cc:270] Start()
1211 20:55:17.665429 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497084343 usec, L: 0
1211 20:55:17.669477 (+ 4048us) write_op.cc:270] Start()
1211 20:55:17.669502 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497088737 usec, L: 0
1211 20:55:17.684636 (+ 15134us) write_op.cc:270] Start()
1211 20:55:17.684661 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497111674 usec, L: 0
1211 20:55:17.684823 (+ 162us) write_op.cc:270] Start()
1211 20:55:17.684844 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497134542 usec, L: 0
1211 20:55:17.694988 (+ 10144us) write_op.cc:270] Start()
1211 20:55:17.695012 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497139708 usec, L: 0
1211 20:55:17.696185 (+ 1173us) write_op.cc:270] Start()
1211 20:55:17.696207 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497142337 usec, L: 0
1211 20:55:17.696353 (+ 146us) write_op.cc:270] Start()
1211 20:55:17.696373 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497162717 usec, L: 0
1211 20:55:17.711908 (+ 15535us) write_op.cc:270] Start()
1211 20:55:17.711934 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497182102 usec, L: 0
1211 20:55:17.718252 (+ 6318us) write_op.cc:270] Start()
1211 20:55:17.718278 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497183100 usec, L: 0
1211 20:55:17.718437 (+ 159us) write_op.cc:270] Start()
1211 20:55:17.718457 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497216721 usec, L: 0
1211 20:55:17.727598 (+ 9141us) write_op.cc:270] Start()
1211 20:55:17.727625 (+ 27us) write_op.cc:276] Timestamp: P: 1765486497225892 usec, L: 0
1211 20:55:17.730310 (+ 2685us) write_op.cc:270] Start()
1211 20:55:17.730339 (+ 29us) write_op.cc:276] Timestamp: P: 1765486497229405 usec, L: 0
1211 20:55:17.730500 (+ 161us) write_op.cc:270] Start()
1211 20:55:17.730520 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497235478 usec, L: 0
1211 20:55:17.730670 (+ 150us) write_op.cc:270] Start()
1211 20:55:17.730689 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497259513 usec, L: 0
1211 20:55:17.744210 (+ 13521us) write_op.cc:270] Start()
1211 20:55:17.744236 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497293729 usec, L: 0
1211 20:55:17.747991 (+ 3755us) write_op.cc:270] Start()
1211 20:55:17.749381 (+ 1390us) write_op.cc:276] Timestamp: P: 1765486497296853 usec, L: 0
1211 20:55:17.752497 (+ 3116us) write_op.cc:270] Start()
1211 20:55:17.752522 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497304831 usec, L: 0
1211 20:55:17.756213 (+ 3691us) write_op.cc:270] Start()
1211 20:55:17.756230 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497334553 usec, L: 0
1211 20:55:17.759721 (+ 3491us) write_op.cc:270] Start()
1211 20:55:17.759745 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497349131 usec, L: 0
1211 20:55:17.759901 (+ 156us) write_op.cc:270] Start()
1211 20:55:17.759925 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497353149 usec, L: 0
1211 20:55:17.760119 (+ 194us) write_op.cc:270] Start()
1211 20:55:17.760152 (+ 33us) write_op.cc:276] Timestamp: P: 1765486497363370 usec, L: 0
1211 20:55:17.781401 (+ 21249us) write_op.cc:270] Start()
1211 20:55:17.781420 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497376619 usec, L: 0
1211 20:55:17.781544 (+ 124us) write_op.cc:270] Start()
1211 20:55:17.781563 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497398955 usec, L: 0
1211 20:55:17.782371 (+ 808us) write_op.cc:270] Start()
1211 20:55:17.782396 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497412277 usec, L: 0
1211 20:55:17.785332 (+ 2936us) write_op.cc:270] Start()
1211 20:55:17.785354 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497422037 usec, L: 0
1211 20:55:17.785503 (+ 149us) write_op.cc:270] Start()
1211 20:55:17.785524 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497439271 usec, L: 0
1211 20:55:17.785672 (+ 148us) write_op.cc:270] Start()
1211 20:55:17.785694 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497444322 usec, L: 0
1211 20:55:17.785853 (+ 159us) write_op.cc:270] Start()
1211 20:55:17.785877 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497448866 usec, L: 0
1211 20:55:17.786074 (+ 197us) write_op.cc:270] Start()
1211 20:55:17.786097 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497460615 usec, L: 0
1211 20:55:17.786265 (+ 168us) write_op.cc:270] Start()
1211 20:55:17.786287 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497488078 usec, L: 0
1211 20:55:17.786424 (+ 137us) write_op.cc:270] Start()
1211 20:55:17.786446 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497495645 usec, L: 0
1211 20:55:17.789148 (+ 2702us) write_op.cc:270] Start()
1211 20:55:17.789175 (+ 27us) write_op.cc:276] Timestamp: P: 1765486497502629 usec, L: 0
1211 20:55:17.818437 (+ 29262us) write_op.cc:270] Start()
1211 20:55:17.818461 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497518858 usec, L: 0
1211 20:55:17.820766 (+ 2305us) write_op.cc:270] Start()
1211 20:55:17.820786 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497538095 usec, L: 0
1211 20:55:17.830424 (+ 9638us) write_op.cc:270] Start()
1211 20:55:17.830446 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497546942 usec, L: 0
1211 20:55:17.830591 (+ 145us) write_op.cc:270] Start()
1211 20:55:17.830611 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497552846 usec, L: 0
1211 20:55:17.833387 (+ 2776us) write_op.cc:270] Start()
1211 20:55:17.833434 (+ 47us) write_op.cc:276] Timestamp: P: 1765486497559777 usec, L: 0
1211 20:55:17.833645 (+ 211us) write_op.cc:270] Start()
1211 20:55:17.833667 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497587030 usec, L: 0
1211 20:55:17.833856 (+ 189us) write_op.cc:270] Start()
1211 20:55:17.833875 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497593033 usec, L: 0
1211 20:55:17.834048 (+ 173us) write_op.cc:270] Start()
1211 20:55:17.834070 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497610845 usec, L: 0
1211 20:55:17.834241 (+ 171us) write_op.cc:270] Start()
1211 20:55:17.834260 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497622746 usec, L: 0
1211 20:55:17.834551 (+ 291us) write_op.cc:270] Start()
1211 20:55:17.834574 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497644303 usec, L: 0
1211 20:55:17.834768 (+ 194us) write_op.cc:270] Start()
1211 20:55:17.834790 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497654100 usec, L: 0
1211 20:55:17.834953 (+ 163us) write_op.cc:270] Start()
1211 20:55:17.834972 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497665544 usec, L: 0
1211 20:55:17.835139 (+ 167us) write_op.cc:270] Start()
1211 20:55:17.835157 (+ 18us) write_op.cc:276] Timestamp: P: 1765486497682525 usec, L: 0
1211 20:55:17.835292 (+ 135us) write_op.cc:270] Start()
1211 20:55:17.835309 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497687570 usec, L: 0
1211 20:55:17.835446 (+ 137us) write_op.cc:270] Start()
1211 20:55:17.835463 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497701498 usec, L: 0
1211 20:55:17.835624 (+ 161us) write_op.cc:270] Start()
1211 20:55:17.835641 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497713225 usec, L: 0
1211 20:55:17.835769 (+ 128us) write_op.cc:270] Start()
1211 20:55:17.835793 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497739313 usec, L: 0
1211 20:55:17.835935 (+ 142us) write_op.cc:270] Start()
1211 20:55:17.835959 (+ 24us) write_op.cc:276] Timestamp: P: 1765486497747508 usec, L: 0
1211 20:55:17.836103 (+ 144us) write_op.cc:270] Start()
1211 20:55:17.836162 (+ 59us) write_op.cc:276] Timestamp: P: 1765486497763257 usec, L: 0
1211 20:55:17.836418 (+ 256us) write_op.cc:270] Start()
1211 20:55:17.836439 (+ 21us) write_op.cc:276] Timestamp: P: 1765486497777847 usec, L: 0
1211 20:55:17.836638 (+ 199us) write_op.cc:270] Start()
1211 20:55:17.836684 (+ 46us) write_op.cc:276] Timestamp: P: 1765486497790061 usec, L: 0
1211 20:55:17.836860 (+ 176us) write_op.cc:270] Start()
1211 20:55:17.836879 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497800316 usec, L: 0
1211 20:55:17.837013 (+ 134us) write_op.cc:270] Start()
1211 20:55:17.837033 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497813839 usec, L: 0
1211 20:55:17.837174 (+ 141us) write_op.cc:270] Start()
1211 20:55:17.837197 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497832810 usec, L: 0
1211 20:55:17.837351 (+ 154us) write_op.cc:270] Start()
1211 20:55:17.837390 (+ 39us) write_op.cc:276] Timestamp: P: 1765486497837055 usec, L: 0
1211 20:55:17.837575 (+ 185us) write_op.cc:270] Start()
1211 20:55:17.837595 (+ 20us) write_op.cc:276] Timestamp: P: 1765486497854420 usec, L: 0
1211 20:55:17.874346 (+ 36751us) write_op.cc:270] Start()
1211 20:55:17.874373 (+ 27us) write_op.cc:276] Timestamp: P: 1765486497868842 usec, L: 0
1211 20:55:17.874537 (+ 164us) write_op.cc:270] Start()
1211 20:55:17.874556 (+ 19us) write_op.cc:276] Timestamp: P: 1765486497882129 usec, L: 0
1211 20:55:17.874735 (+ 179us) write_op.cc:270] Start()
1211 20:55:17.874761 (+ 26us) write_op.cc:276] Timestamp: P: 1765486497887789 usec, L: 0
1211 20:55:17.874941 (+ 180us) write_op.cc:270] Start()
1211 20:55:17.874966 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497900855 usec, L: 0
1211 20:55:17.875151 (+ 185us) write_op.cc:270] Start()
1211 20:55:17.875174 (+ 23us) write_op.cc:276] Timestamp: P: 1765486497930529 usec, L: 0
1211 20:55:17.875343 (+ 169us) write_op.cc:270] Start()
1211 20:55:17.875368 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497951831 usec, L: 0
1211 20:55:17.875531 (+ 163us) write_op.cc:270] Start()
1211 20:55:17.875553 (+ 22us) write_op.cc:276] Timestamp: P: 1765486497958146 usec, L: 0
1211 20:55:17.875685 (+ 132us) write_op.cc:270] Start()
1211 20:55:17.875702 (+ 17us) write_op.cc:276] Timestamp: P: 1765486497960086 usec, L: 0
1211 20:55:17.879393 (+ 3691us) write_op.cc:270] Start()
1211 20:55:17.879418 (+ 25us) write_op.cc:276] Timestamp: P: 1765486497985049 usec, L: 0
1211 20:55:17.879589 (+ 171us) write_op.cc:270] Start()
1211 20:55:17.879610 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498007707 usec, L: 0
1211 20:55:17.879769 (+ 159us) write_op.cc:270] Start()
1211 20:55:17.881047 (+ 1278us) write_op.cc:276] Timestamp: P: 1765486498033687 usec, L: 0
1211 20:55:17.881233 (+ 186us) write_op.cc:270] Start()
1211 20:55:17.881254 (+ 21us) write_op.cc:276] Timestamp: P: 1765486498036795 usec, L: 0
1211 20:55:17.881389 (+ 135us) write_op.cc:270] Start()
1211 20:55:17.881408 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498043170 usec, L: 0
1211 20:55:17.881522 (+ 114us) write_op.cc:270] Start()
1211 20:55:17.881539 (+ 17us) write_op.cc:276] Timestamp: P: 1765486498047561 usec, L: 0
1211 20:55:17.881652 (+ 113us) write_op.cc:270] Start()
1211 20:55:17.881669 (+ 17us) write_op.cc:276] Timestamp: P: 1765486498080937 usec, L: 0
1211 20:55:17.881798 (+ 129us) write_op.cc:270] Start()
1211 20:55:17.881815 (+ 17us) write_op.cc:276] Timestamp: P: 1765486498090561 usec, L: 0
1211 20:55:17.881962 (+ 147us) write_op.cc:270] Start()
1211 20:55:17.881980 (+ 18us) write_op.cc:276] Timestamp: P: 1765486498111709 usec, L: 0
1211 20:55:17.887063 (+ 5083us) write_op.cc:270] Start()
1211 20:55:17.887089 (+ 26us) write_op.cc:276] Timestamp: P: 1765486498126574 usec, L: 0
1211 20:55:17.887259 (+ 170us) write_op.cc:270] Start()
1211 20:55:17.887282 (+ 23us) write_op.cc:276] Timestamp: P: 1765486498132853 usec, L: 0
1211 20:55:17.887420 (+ 138us) write_op.cc:270] Start()
1211 20:55:17.887440 (+ 20us) write_op.cc:276] Timestamp: P: 1765486498136050 usec, L: 0
1211 20:55:17.887581 (+ 141us) write_op.cc:270] Start()
1211 20:55:17.887600 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498156833 usec, L: 0
1211 20:55:17.887753 (+ 153us) write_op.cc:270] Start()
1211 20:55:17.887772 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498181182 usec, L: 0
1211 20:55:17.887904 (+ 132us) write_op.cc:270] Start()
1211 20:55:17.887923 (+ 19us) write_op.cc:276] Timestamp: P: 1765486498184674 usec, L: 0
1211 20:55:17.888047 (+ 124us) write_op.cc:270] Start()
1211 20:55:17.888067 (+ 20us) write_op.cc:276] Timestamp: P: 1765
I20251211 20:55:23.264588 6776 ts_manager.cc:284] Unset tserver state for cd482952c9fd4a1fb75c31f128338e89 from MAINTENANCE_MODE
I20251211 20:55:23.272629 6703 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:55:23.469041 7100 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:23.469820 7100 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } }
I20251211 20:55:23.475909 6277 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1640 } } ignore_live_leader: false dest_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" is_pre_election: true
I20251211 20:55:23.476366 7100 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:23.486402 6405 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "4927adf430d04b6199dbbfaf1de1c16d" candidate_uuid: "cd482952c9fd4a1fb75c31f128338e89" candidate_term: 2 candidate_status { last_received { term: 1 index: 1640 } } ignore_live_leader: false dest_uuid: "d60a4040120049e48295adbd7c33e3e4" is_pre_election: true
I20251211 20:55:23.490700 6872 leader_election.cc:304] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cd482952c9fd4a1fb75c31f128338e89; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, d60a4040120049e48295adbd7c33e3e4
I20251211 20:55:23.505311 7100 raft_consensus.cc:2749] T 4927adf430d04b6199dbbfaf1de1c16d P cd482952c9fd4a1fb75c31f128338e89 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:23.632262 6453 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
W20251211 20:55:23.684671 6344 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.5.211.254:40613, user_credentials={real_user=slave}} blocked reactor thread for 52608.9us
I20251211 20:55:23.917609 7006 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:55:24.007524 6323 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
W20251211 20:55:25.338338 6344 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.211.193:34639, user_credentials={real_user=slave}} blocked reactor thread for 62112.9us
I20251211 20:55:26.751922 6776 ts_manager.cc:295] Set tserver state for cd482952c9fd4a1fb75c31f128338e89 to MAINTENANCE_MODE
I20251211 20:55:26.753008 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 6852
I20251211 20:55:26.756986 7000 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:55:27.404214 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 6852
W20251211 20:55:27.526741 6214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.211.193:34639 (error 108) [suppressed 40 similar messages]
W20251211 20:55:27.531064 6344 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.211.193:34639 (error 108) [suppressed 47 similar messages]
W20251211 20:55:27.538618 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:27.539093 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:27.539198 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:27.563141 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:27.563539 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:27.563684 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:27.949810 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:28.002478 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:28.019502 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:28.030927 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:28.052052 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:28.100472 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:28.486240 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:28.515519 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:28.540393 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:28.552659 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:28.588244 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:28.595814 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20251211 20:55:28.709650 7086 consensus_queue.cc:579] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.015s)
I20251211 20:55:28.752997 7111 consensus_queue.cc:579] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.028s)
I20251211 20:55:28.762949 7052 consensus_queue.cc:579] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.005s)
I20251211 20:55:28.797129 7052 consensus_queue.cc:579] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.038s)
I20251211 20:55:28.812393 7112 consensus_queue.cc:579] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.052s)
I20251211 20:55:28.814734 7135 consensus_queue.cc:579] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.090s)
W20251211 20:55:29.020674 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:55:29.052897 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:55:29.076468 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20251211 20:55:29.094743 6776 ts_manager.cc:284] Unset tserver state for cd482952c9fd4a1fb75c31f128338e89 from MAINTENANCE_MODE
W20251211 20:55:29.097342 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:55:29.128710 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:55:29.201021 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20251211 20:55:29.290910 6703 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
W20251211 20:55:29.510528 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:55:29.525751 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:55:29.586889 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:55:29.645475 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:55:29.666816 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:55:29.770653 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20251211 20:55:29.809126 6323 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:55:29.839938 6276 consensus_queue.cc:237] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2004, Committed index: 2004, Last appended: 1.2006, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2007 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:29.839941 6277 consensus_queue.cc:237] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2005, Committed index: 2005, Last appended: 1.2007, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:29.845484 6275 consensus_queue.cc:237] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2005, Committed index: 2005, Last appended: 1.2007, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:29.847226 6453 heartbeater.cc:507] Master 127.5.211.254:40613 requested a full tablet report, sending...
I20251211 20:55:29.852927 6405 raft_consensus.cc:1275] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Refusing update from remote peer 4acae5cddb8e4a9b8777025ea93a4b77: Log matching property violated. Preceding OpId in replica: term: 1 index: 2005. Preceding OpId from leader: term: 1 index: 2007. (index mismatch)
I20251211 20:55:29.860298 6869 consensus_queue.cc:1048] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2007, Last known committed idx: 2004, Time since last communication: 0.000s
I20251211 20:55:29.860909 6405 raft_consensus.cc:1275] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Refusing update from remote peer 4acae5cddb8e4a9b8777025ea93a4b77: Log matching property violated. Preceding OpId in replica: term: 1 index: 2006. Preceding OpId from leader: term: 1 index: 2008. (index mismatch)
W20251211 20:55:29.862445 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251211 20:55:29.862737 6407 raft_consensus.cc:1275] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Refusing update from remote peer 4acae5cddb8e4a9b8777025ea93a4b77: Log matching property violated. Preceding OpId in replica: term: 1 index: 2006. Preceding OpId from leader: term: 1 index: 2008. (index mismatch)
W20251211 20:55:29.863276 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:29.863818 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251211 20:55:29.864670 6869 consensus_queue.cc:1048] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2008, Last known committed idx: 2005, Time since last communication: 0.000s
I20251211 20:55:29.865861 7138 consensus_queue.cc:1048] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2008, Last known committed idx: 2005, Time since last communication: 0.000s
I20251211 20:55:29.867374 7137 raft_consensus.cc:2955] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 LEADER]: Committing config change with OpId 1.2007: config changed from index -1 to 2007, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2007 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:29.882890 7138 raft_consensus.cc:2955] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 LEADER]: Committing config change with OpId 1.2008: config changed from index -1 to 2008, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:29.882846 6776 catalog_manager.cc:5654] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 reported cstate change: config changed from index -1 to 2007, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New cstate: current_term: 1 leader_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" committed_config { opid_index: 2007 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251211 20:55:29.888890 6407 raft_consensus.cc:2955] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2007: config changed from index -1 to 2007, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2007 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:29.890645 7137 raft_consensus.cc:2955] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 LEADER]: Committing config change with OpId 1.2008: config changed from index -1 to 2008, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:29.896739 6406 raft_consensus.cc:2955] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2008: config changed from index -1 to 2008, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:29.895104 6405 raft_consensus.cc:2955] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Committing config change with OpId 1.2008: config changed from index -1 to 2008, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:29.904841 6775 catalog_manager.cc:5654] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 reported cstate change: config changed from index -1 to 2008, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New cstate: current_term: 1 leader_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" committed_config { opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:29.910609 6406 consensus_queue.cc:237] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2009, Committed index: 2009, Last appended: 1.2009, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:29.910583 6408 consensus_queue.cc:237] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2009, Committed index: 2009, Last appended: 1.2009, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:29.912475 6404 consensus_queue.cc:237] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2009, Committed index: 2009, Last appended: 1.2009, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:29.925256 6763 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a9dbb37290964ca7ba650d2de1404245 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251211 20:55:29.925864 6763 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet d915f836ed2c479c9d6623e11266ed55 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251211 20:55:29.926153 6763 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 4927adf430d04b6199dbbfaf1de1c16d with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
W20251211 20:55:29.929900 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d915f836ed2c479c9d6623e11266ed55. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:29.930536 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a9dbb37290964ca7ba650d2de1404245. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:29.930943 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4927adf430d04b6199dbbfaf1de1c16d. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:55:29.950379 7152 ts_tablet_manager.cc:933] T 4927adf430d04b6199dbbfaf1de1c16d P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Initiating tablet copy from peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:55:29.950636 7153 ts_tablet_manager.cc:933] T d915f836ed2c479c9d6623e11266ed55 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Initiating tablet copy from peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:55:29.954770 7153 tablet_copy_client.cc:323] T d915f836ed2c479c9d6623e11266ed55 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Beginning tablet copy session from remote peer at address 127.5.211.194:45317
I20251211 20:55:29.963419 7152 tablet_copy_client.cc:323] T 4927adf430d04b6199dbbfaf1de1c16d P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Beginning tablet copy session from remote peer at address 127.5.211.194:45317
I20251211 20:55:29.980434 6297 tablet_copy_service.cc:140] P 4acae5cddb8e4a9b8777025ea93a4b77: Received BeginTabletCopySession request for tablet 4927adf430d04b6199dbbfaf1de1c16d from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 ({username='slave'} at 127.5.211.196:43939)
I20251211 20:55:29.980706 6297 tablet_copy_service.cc:161] P 4acae5cddb8e4a9b8777025ea93a4b77: Beginning new tablet copy session on tablet 4927adf430d04b6199dbbfaf1de1c16d from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 at {username='slave'} at 127.5.211.196:43939: session id = 5c8c4cb5d8ae430e9d6334daaa9e7b41-4927adf430d04b6199dbbfaf1de1c16d
I20251211 20:55:29.984658 6297 tablet_copy_source_session.cc:215] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77: Tablet Copy: opened 0 blocks and 1 log segments
I20251211 20:55:29.980434 6298 tablet_copy_service.cc:140] P 4acae5cddb8e4a9b8777025ea93a4b77: Received BeginTabletCopySession request for tablet d915f836ed2c479c9d6623e11266ed55 from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 ({username='slave'} at 127.5.211.196:43939)
I20251211 20:55:29.987358 6298 tablet_copy_service.cc:161] P 4acae5cddb8e4a9b8777025ea93a4b77: Beginning new tablet copy session on tablet d915f836ed2c479c9d6623e11266ed55 from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 at {username='slave'} at 127.5.211.196:43939: session id = 5c8c4cb5d8ae430e9d6334daaa9e7b41-d915f836ed2c479c9d6623e11266ed55
I20251211 20:55:29.990090 6298 tablet_copy_source_session.cc:215] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77: Tablet Copy: opened 0 blocks and 1 log segments
I20251211 20:55:29.991564 7152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4927adf430d04b6199dbbfaf1de1c16d. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:29.991994 7153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d915f836ed2c479c9d6623e11266ed55. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:30.002622 6277 raft_consensus.cc:1275] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Refusing update from remote peer d60a4040120049e48295adbd7c33e3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2009. Preceding OpId from leader: term: 1 index: 2010. (index mismatch)
I20251211 20:55:30.002889 6275 raft_consensus.cc:1275] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Refusing update from remote peer d60a4040120049e48295adbd7c33e3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2009. Preceding OpId from leader: term: 1 index: 2010. (index mismatch)
I20251211 20:55:30.003849 6275 raft_consensus.cc:1275] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Refusing update from remote peer d60a4040120049e48295adbd7c33e3e4: Log matching property violated. Preceding OpId in replica: term: 1 index: 2009. Preceding OpId from leader: term: 1 index: 2010. (index mismatch)
I20251211 20:55:30.004117 7136 consensus_queue.cc:1048] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2010, Last known committed idx: 2009, Time since last communication: 0.000s
I20251211 20:55:30.004489 7110 consensus_queue.cc:1048] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2010, Last known committed idx: 2009, Time since last communication: 0.000s
W20251211 20:55:30.005191 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:30.005378 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:30.005555 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251211 20:55:30.007042 7153 tablet_copy_client.cc:806] T d915f836ed2c479c9d6623e11266ed55 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 0 data blocks...
I20251211 20:55:30.007509 7153 tablet_copy_client.cc:670] T d915f836ed2c479c9d6623e11266ed55 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 1 WAL segments...
I20251211 20:55:30.007136 7152 tablet_copy_client.cc:806] T 4927adf430d04b6199dbbfaf1de1c16d P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 0 data blocks...
I20251211 20:55:30.008394 7152 tablet_copy_client.cc:670] T 4927adf430d04b6199dbbfaf1de1c16d P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 1 WAL segments...
I20251211 20:55:30.013561 6776 catalog_manager.cc:5654] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 reported cstate change: config changed from index -1 to 2008, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New cstate: current_term: 1 leader_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" committed_config { opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251211 20:55:30.021417 7153 tablet_copy_client.cc:538] T d915f836ed2c479c9d6623e11266ed55 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251211 20:55:30.025170 7110 consensus_queue.cc:1048] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2010, Last known committed idx: 2009, Time since last communication: 0.000s
I20251211 20:55:30.026350 7136 raft_consensus.cc:2955] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [term 1 LEADER]: Committing config change with OpId 1.2010: config changed from index -1 to 2010, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:30.026350 7085 raft_consensus.cc:2955] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [term 1 LEADER]: Committing config change with OpId 1.2010: config changed from index -1 to 2010, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
W20251211 20:55:30.029050 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b788bd87c9a4ca8a88d144e2af0c6c9. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:30.029177 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ce1e22ac4b814a508b20688b15d0c959. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:30.029269 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e7ef1dab1f4c47379f2756f162302ecc. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:55:30.029986 6277 raft_consensus.cc:2955] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Committing config change with OpId 1.2010: config changed from index -1 to 2010, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:30.032001 7135 raft_consensus.cc:2955] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [term 1 LEADER]: Committing config change with OpId 1.2010: config changed from index -1 to 2010, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:30.033866 7153 tablet_bootstrap.cc:492] T d915f836ed2c479c9d6623e11266ed55 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Bootstrap starting.
I20251211 20:55:30.034018 6276 raft_consensus.cc:2955] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Committing config change with OpId 1.2010: config changed from index -1 to 2010, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:30.038390 6763 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet ce1e22ac4b814a508b20688b15d0c959 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251211 20:55:30.038676 6763 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9 with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251211 20:55:30.040403 6776 catalog_manager.cc:5654] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 reported cstate change: config changed from index -1 to 2010, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New cstate: current_term: 1 leader_uuid: "d60a4040120049e48295adbd7c33e3e4" committed_config { opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:30.044469 6763 catalog_manager.cc:5167] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet e7ef1dab1f4c47379f2756f162302ecc with cas_config_opid_index -1: ChangeConfig:ADD_PEER:NON_VOTER succeeded (attempt 1)
I20251211 20:55:30.046594 6776 catalog_manager.cc:5654] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 reported cstate change: config changed from index -1 to 2010, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New cstate: current_term: 1 leader_uuid: "d60a4040120049e48295adbd7c33e3e4" committed_config { opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:30.031702 6275 raft_consensus.cc:2955] T ce1e22ac4b814a508b20688b15d0c959 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Committing config change with OpId 1.2010: config changed from index -1 to 2010, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New config: { opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } } }
I20251211 20:55:30.048928 6775 catalog_manager.cc:5654] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 reported cstate change: config changed from index -1 to 2010, NON_VOTER 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196) added. New cstate: current_term: 1 leader_uuid: "d60a4040120049e48295adbd7c33e3e4" committed_config { opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20251211 20:55:30.057611 7152 tablet_copy_client.cc:538] T 4927adf430d04b6199dbbfaf1de1c16d P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251211 20:55:30.069252 7159 ts_tablet_manager.cc:933] T a9dbb37290964ca7ba650d2de1404245 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Initiating tablet copy from peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:55:30.106412 7152 tablet_bootstrap.cc:492] T 4927adf430d04b6199dbbfaf1de1c16d P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Bootstrap starting.
I20251211 20:55:30.119287 7162 ts_tablet_manager.cc:933] T e7ef1dab1f4c47379f2756f162302ecc P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Initiating tablet copy from peer d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:30.126266 7158 ts_tablet_manager.cc:933] T ce1e22ac4b814a508b20688b15d0c959 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Initiating tablet copy from peer d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:30.134598 7159 tablet_copy_client.cc:323] T a9dbb37290964ca7ba650d2de1404245 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Beginning tablet copy session from remote peer at address 127.5.211.194:45317
I20251211 20:55:30.136283 6297 tablet_copy_service.cc:140] P 4acae5cddb8e4a9b8777025ea93a4b77: Received BeginTabletCopySession request for tablet a9dbb37290964ca7ba650d2de1404245 from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 ({username='slave'} at 127.5.211.196:43939)
I20251211 20:55:30.136498 6297 tablet_copy_service.cc:161] P 4acae5cddb8e4a9b8777025ea93a4b77: Beginning new tablet copy session on tablet a9dbb37290964ca7ba650d2de1404245 from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 at {username='slave'} at 127.5.211.196:43939: session id = 5c8c4cb5d8ae430e9d6334daaa9e7b41-a9dbb37290964ca7ba650d2de1404245
I20251211 20:55:30.144546 6297 tablet_copy_source_session.cc:215] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77: Tablet Copy: opened 0 blocks and 1 log segments
I20251211 20:55:30.146554 7159 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9dbb37290964ca7ba650d2de1404245. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:30.153012 7159 tablet_copy_client.cc:806] T a9dbb37290964ca7ba650d2de1404245 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 0 data blocks...
I20251211 20:55:30.154667 7161 ts_tablet_manager.cc:933] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Initiating tablet copy from peer d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:30.158016 7159 tablet_copy_client.cc:670] T a9dbb37290964ca7ba650d2de1404245 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 1 WAL segments...
I20251211 20:55:30.162276 7162 tablet_copy_client.cc:323] T e7ef1dab1f4c47379f2756f162302ecc P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Beginning tablet copy session from remote peer at address 127.5.211.195:46117
I20251211 20:55:30.163928 7161 tablet_copy_client.cc:323] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Beginning tablet copy session from remote peer at address 127.5.211.195:46117
I20251211 20:55:30.181157 7159 tablet_copy_client.cc:538] T a9dbb37290964ca7ba650d2de1404245 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251211 20:55:30.185029 7159 tablet_bootstrap.cc:492] T a9dbb37290964ca7ba650d2de1404245 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Bootstrap starting.
I20251211 20:55:30.184929 6428 tablet_copy_service.cc:140] P d60a4040120049e48295adbd7c33e3e4: Received BeginTabletCopySession request for tablet e7ef1dab1f4c47379f2756f162302ecc from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 ({username='slave'} at 127.5.211.196:46873)
I20251211 20:55:30.185195 6428 tablet_copy_service.cc:161] P d60a4040120049e48295adbd7c33e3e4: Beginning new tablet copy session on tablet e7ef1dab1f4c47379f2756f162302ecc from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 at {username='slave'} at 127.5.211.196:46873: session id = 5c8c4cb5d8ae430e9d6334daaa9e7b41-e7ef1dab1f4c47379f2756f162302ecc
I20251211 20:55:30.188782 6428 tablet_copy_source_session.cc:215] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20251211 20:55:30.188992 6427 tablet_copy_service.cc:140] P d60a4040120049e48295adbd7c33e3e4: Received BeginTabletCopySession request for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9 from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 ({username='slave'} at 127.5.211.196:46873)
I20251211 20:55:30.189203 6427 tablet_copy_service.cc:161] P d60a4040120049e48295adbd7c33e3e4: Beginning new tablet copy session on tablet 0b788bd87c9a4ca8a88d144e2af0c6c9 from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 at {username='slave'} at 127.5.211.196:46873: session id = 5c8c4cb5d8ae430e9d6334daaa9e7b41-0b788bd87c9a4ca8a88d144e2af0c6c9
I20251211 20:55:30.192771 6427 tablet_copy_source_session.cc:215] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20251211 20:55:30.204973 7158 tablet_copy_client.cc:323] T ce1e22ac4b814a508b20688b15d0c959 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Beginning tablet copy session from remote peer at address 127.5.211.195:46117
I20251211 20:55:30.214708 6427 tablet_copy_service.cc:140] P d60a4040120049e48295adbd7c33e3e4: Received BeginTabletCopySession request for tablet ce1e22ac4b814a508b20688b15d0c959 from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 ({username='slave'} at 127.5.211.196:46873)
I20251211 20:55:30.214861 6427 tablet_copy_service.cc:161] P d60a4040120049e48295adbd7c33e3e4: Beginning new tablet copy session on tablet ce1e22ac4b814a508b20688b15d0c959 from peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 at {username='slave'} at 127.5.211.196:46873: session id = 5c8c4cb5d8ae430e9d6334daaa9e7b41-ce1e22ac4b814a508b20688b15d0c959
I20251211 20:55:30.215041 7161 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0b788bd87c9a4ca8a88d144e2af0c6c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:30.216612 6427 tablet_copy_source_session.cc:215] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4: Tablet Copy: opened 0 blocks and 1 log segments
I20251211 20:55:30.217660 7158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ce1e22ac4b814a508b20688b15d0c959. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:30.220089 7162 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e7ef1dab1f4c47379f2756f162302ecc. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:30.229255 7162 tablet_copy_client.cc:806] T e7ef1dab1f4c47379f2756f162302ecc P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 0 data blocks...
I20251211 20:55:30.233592 7162 tablet_copy_client.cc:670] T e7ef1dab1f4c47379f2756f162302ecc P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 1 WAL segments...
I20251211 20:55:30.233744 7161 tablet_copy_client.cc:806] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 0 data blocks...
I20251211 20:55:30.234081 7161 tablet_copy_client.cc:670] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 1 WAL segments...
I20251211 20:55:30.250458 7158 tablet_copy_client.cc:806] T ce1e22ac4b814a508b20688b15d0c959 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 0 data blocks...
I20251211 20:55:30.250764 7158 tablet_copy_client.cc:670] T ce1e22ac4b814a508b20688b15d0c959 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Starting download of 1 WAL segments...
I20251211 20:55:30.255479 7161 tablet_copy_client.cc:538] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251211 20:55:30.283677 7162 tablet_copy_client.cc:538] T e7ef1dab1f4c47379f2756f162302ecc P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251211 20:55:30.296721 7158 tablet_copy_client.cc:538] T ce1e22ac4b814a508b20688b15d0c959 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20251211 20:55:30.300885 7161 tablet_bootstrap.cc:492] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Bootstrap starting.
I20251211 20:55:30.303105 7162 tablet_bootstrap.cc:492] T e7ef1dab1f4c47379f2756f162302ecc P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Bootstrap starting.
I20251211 20:55:30.307997 7158 tablet_bootstrap.cc:492] T ce1e22ac4b814a508b20688b15d0c959 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Bootstrap starting.
W20251211 20:55:30.332398 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.358078 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.361076 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.441011 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.475426 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.476464 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.477278 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.487044 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.635150 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.680933 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.724561 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.734654 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20251211 20:55:30.822461 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:30.854270 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:30.895780 6214 consensus_peers.cc:597] T a9dbb37290964ca7ba650d2de1404245 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:30.926420 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:30.938076 6214 consensus_peers.cc:597] T 4927adf430d04b6199dbbfaf1de1c16d P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:30.956918 6214 consensus_peers.cc:597] T d915f836ed2c479c9d6623e11266ed55 P 4acae5cddb8e4a9b8777025ea93a4b77 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:30.980175 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:31.000355 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20251211 20:55:31.032672 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 6196
I20251211 20:55:31.036630 6318 generic_service.cc:196] Checking for leaks (request via RPC)
W20251211 20:55:31.146854 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
W20251211 20:55:31.227596 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20251211 20:55:31.253890 7153 log.cc:826] T d915f836ed2c479c9d6623e11266ed55 P 5c8c4cb5d8ae430e9d6334daaa9e7b41: Log is configured to *not* fsync() on all Append() calls
W20251211 20:55:31.271343 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20251211 20:55:31.282768 7110 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:31.282972 7110 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:31.283643 7110 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
W20251211 20:55:31.284888 6344 leader_election.cc:336] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111)
I20251211 20:55:31.342782 7110 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:31.342954 7110 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:31.343600 7110 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
W20251211 20:55:31.344925 6344 leader_election.cc:336] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111)
I20251211 20:55:31.358062 7110 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:31.358227 7110 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2007 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:31.358824 7110 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
W20251211 20:55:31.360013 6344 leader_election.cc:336] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111)
W20251211 20:55:31.369467 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:55:31.492904 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:55:31.529326 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20251211 20:55:31.642349 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 16: this message will repeat every 5th retry.
I20251211 20:55:31.718366 7134 raft_consensus.cc:493] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:31.718575 7134 raft_consensus.cc:515] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:31.721166 7134 leader_election.cc:290] T e7ef1dab1f4c47379f2756f162302ecc P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
W20251211 20:55:31.727520 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 16: this message will repeat every 5th retry.
I20251211 20:55:31.769150 6869 raft_consensus.cc:493] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d60a4040120049e48295adbd7c33e3e4)
I20251211 20:55:31.769315 6869 raft_consensus.cc:515] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2010 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:31.769893 6869 leader_election.cc:290] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P 4acae5cddb8e4a9b8777025ea93a4b77 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), d60a4040120049e48295adbd7c33e3e4 (127.5.211.195:46117)
I20251211 20:55:31.772845 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 6196
W20251211 20:55:31.781543 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer 5c8c4cb5d8ae430e9d6334daaa9e7b41 (127.5.211.196:35667): Couldn't send request to peer 5c8c4cb5d8ae430e9d6334daaa9e7b41. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 16: this message will repeat every 5th retry.
W20251211 20:55:31.907423 6344 leader_election.cc:336] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317): Timed out: RequestConsensusVote RPC to 127.5.211.194:45317 timed out after 0.311s (SENT)
I20251211 20:55:31.907599 6344 leader_election.cc:304] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60a4040120049e48295adbd7c33e3e4; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, cd482952c9fd4a1fb75c31f128338e89
I20251211 20:55:31.907876 7085 raft_consensus.cc:2749] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:31.935249 7085 consensus_queue.cc:579] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.022s)
W20251211 20:55:31.948709 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:55:31.954159 6343 connection.cc:537] server connection from 127.5.211.194:51575 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:55:31.959331 7085 consensus_queue.cc:579] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.048s)
W20251211 20:55:31.962947 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639): Couldn't send request to peer cd482952c9fd4a1fb75c31f128338e89. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:34639: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20251211 20:55:31.965500 6344 connection.cc:537] client connection to 127.5.211.194:45317 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:55:31.965605 6344 leader_election.cc:336] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:55:31.965698 6344 leader_election.cc:304] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60a4040120049e48295adbd7c33e3e4; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, cd482952c9fd4a1fb75c31f128338e89
W20251211 20:55:31.965978 6761 connection.cc:537] server connection from 127.5.211.194:50357 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:55:31.966032 6344 leader_election.cc:336] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317): Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:55:31.966142 6344 leader_election.cc:304] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d60a4040120049e48295adbd7c33e3e4; no voters: 4acae5cddb8e4a9b8777025ea93a4b77, cd482952c9fd4a1fb75c31f128338e89
I20251211 20:55:31.967392 7085 raft_consensus.cc:2749] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20251211 20:55:31.967572 7085 raft_consensus.cc:2749] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20251211 20:55:31.967836 6344 consensus_peers.cc:597] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 -> Peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317): Couldn't send request to peer 4acae5cddb8e4a9b8777025ea93a4b77. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.194:45317: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:31.967947 6344 consensus_peers.cc:597] T 0b788bd87c9a4ca8a88d144e2af0c6c9 P d60a4040120049e48295adbd7c33e3e4 -> Peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317): Couldn't send request to peer 4acae5cddb8e4a9b8777025ea93a4b77. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.194:45317: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:31.968034 6344 consensus_peers.cc:597] T ce1e22ac4b814a508b20688b15d0c959 P d60a4040120049e48295adbd7c33e3e4 -> Peer 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317): Couldn't send request to peer 4acae5cddb8e4a9b8777025ea93a4b77. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.194:45317: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251211 20:55:31.973928 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 6326
I20251211 20:55:31.974934 6448 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:55:32.480239 7187 raft_consensus.cc:493] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:32.480458 7187 raft_consensus.cc:515] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2007 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:32.481184 7187 leader_election.cc:290] T a9dbb37290964ca7ba650d2de1404245 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:55:32.505820 7187 consensus_queue.cc:579] T e7ef1dab1f4c47379f2756f162302ecc P d60a4040120049e48295adbd7c33e3e4 [LEADER]: Leader has been unable to successfully communicate with peer cd482952c9fd4a1fb75c31f128338e89 for more than 2 seconds (2.595s)
I20251211 20:55:32.529578 7187 raft_consensus.cc:493] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:32.529747 7187 raft_consensus.cc:515] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:32.530344 7187 leader_election.cc:290] T d915f836ed2c479c9d6623e11266ed55 P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:55:32.533623 7193 raft_consensus.cc:493] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 4acae5cddb8e4a9b8777025ea93a4b77)
I20251211 20:55:32.533841 7193 raft_consensus.cc:515] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2008 OBSOLETE_local: false peers { permanent_uuid: "cd482952c9fd4a1fb75c31f128338e89" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 34639 } } peers { permanent_uuid: "4acae5cddb8e4a9b8777025ea93a4b77" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45317 } } peers { permanent_uuid: "d60a4040120049e48295adbd7c33e3e4" member_type: VOTER last_known_addr { host: "127.5.211.195" port: 46117 } } peers { permanent_uuid: "5c8c4cb5d8ae430e9d6334daaa9e7b41" member_type: NON_VOTER last_known_addr { host: "127.5.211.196" port: 35667 } attrs { promote: true } }
I20251211 20:55:32.534475 7193 leader_election.cc:290] T 4927adf430d04b6199dbbfaf1de1c16d P d60a4040120049e48295adbd7c33e3e4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cd482952c9fd4a1fb75c31f128338e89 (127.5.211.193:34639), 4acae5cddb8e4a9b8777025ea93a4b77 (127.5.211.194:45317)
I20251211 20:55:32.537918 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 6326
I20251211 20:55:32.802273 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 6538
I20251211 20:55:32.803887 6698 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:55:33.019593 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 6538
I20251211 20:55:33.050917 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 6742
I20251211 20:55:33.051976 6809 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:55:33.155615 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 6742
2025-12-11T20:55:33Z chronyd exiting
[ OK ] MaintenanceModeRF3ITest.TestFailedTServerInMaintenanceModeDoesntRereplicate (43758 ms)
[----------] 1 test from MaintenanceModeRF3ITest (43758 ms total)
[----------] 1 test from RollingRestartArgs/RollingRestartITest
[ RUN ] RollingRestartArgs/RollingRestartITest.TestWorkloads/4
2025-12-11T20:55:33Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-12-11T20:55:33Z Disabled control of system clock
I20251211 20:55:33.248561 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.211.254:43355
--webserver_interface=127.5.211.254
--webserver_port=0
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.211.254:43355
--location_mapping_cmd=/tmp/dist-test-taskrKV6hb/build/asan/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/location-assignment.state --map /L0:4
--master_client_location_assignment_enabled=false with env {}
W20251211 20:55:33.553748 7200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:33.554057 7200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:33.554140 7200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:33.561461 7200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20251211 20:55:33.561530 7200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:33.561581 7200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20251211 20:55:33.561616 7200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20251211 20:55:33.570297 7200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/wal
--location_mapping_cmd=/tmp/dist-test-taskrKV6hb/build/asan/bin/testdata/assign-location.py --state_store=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/location-assignment.state --map /L0:4
--ipki_ca_key_size=768
--master_addresses=127.5.211.254:43355
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.211.254:43355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.211.254
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:33.571645 7200 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:33.573182 7200 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:33.580679 7205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:33.580950 7206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:33.581377 7208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:33.581480 7200 server_base.cc:1047] running on GCE node
I20251211 20:55:33.582190 7200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:33.583340 7200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:33.584537 7200 hybrid_clock.cc:648] HybridClock initialized: now 1765486533584492 us; error 36 us; skew 500 ppm
I20251211 20:55:33.584846 7200 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:33.586743 7200 webserver.cc:492] Webserver started at http://127.5.211.254:45431/ using document root <none> and password file <none>
I20251211 20:55:33.587419 7200 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:33.587515 7200 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:33.587845 7200 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:55:33.590121 7200 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/data/instance:
uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16"
format_stamp: "Formatted at 2025-12-11 20:55:33 on dist-test-slave-sk6m"
I20251211 20:55:33.590840 7200 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/wal/instance:
uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16"
format_stamp: "Formatted at 2025-12-11 20:55:33 on dist-test-slave-sk6m"
I20251211 20:55:33.595557 7200 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20251211 20:55:33.598446 7214 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:33.599527 7200 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20251211 20:55:33.599668 7200 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/wal
uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16"
format_stamp: "Formatted at 2025-12-11 20:55:33 on dist-test-slave-sk6m"
I20251211 20:55:33.599853 7200 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:33.620443 7200 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:33.621224 7200 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:33.621476 7200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:33.638926 7200 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.254:43355
I20251211 20:55:33.638937 7265 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.254:43355 every 8 connection(s)
I20251211 20:55:33.640205 7200 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/data/info.pb
I20251211 20:55:33.644412 7266 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:33.648093 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 7200
I20251211 20:55:33.648300 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/master-0/wal/instance
I20251211 20:55:33.653769 7266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16: Bootstrap starting.
I20251211 20:55:33.657181 7266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16: Neither blocks nor log segments found. Creating new log.
I20251211 20:55:33.658465 7266 log.cc:826] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16: Log is configured to *not* fsync() on all Append() calls
I20251211 20:55:33.660995 7266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16: No bootstrap required, opened a new log
I20251211 20:55:33.665788 7266 raft_consensus.cc:359] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 43355 } }
I20251211 20:55:33.666136 7266 raft_consensus.cc:385] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:55:33.666206 7266 raft_consensus.cc:740] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b3dc1110bfe4a2fb24e89fe2c82fd16, State: Initialized, Role: FOLLOWER
I20251211 20:55:33.666720 7266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 43355 } }
I20251211 20:55:33.666877 7266 raft_consensus.cc:399] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20251211 20:55:33.666958 7266 raft_consensus.cc:493] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20251211 20:55:33.667057 7266 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:55:33.668905 7266 raft_consensus.cc:515] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 43355 } }
I20251211 20:55:33.669277 7266 leader_election.cc:304] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1b3dc1110bfe4a2fb24e89fe2c82fd16; no voters:
I20251211 20:55:33.669795 7266 leader_election.cc:290] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20251211 20:55:33.669943 7271 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:55:33.671293 7271 raft_consensus.cc:697] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [term 1 LEADER]: Becoming Leader. State: Replica: 1b3dc1110bfe4a2fb24e89fe2c82fd16, State: Running, Role: LEADER
I20251211 20:55:33.671732 7266 sys_catalog.cc:565] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [sys.catalog]: configured and running, proceeding with master startup.
I20251211 20:55:33.671726 7271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 43355 } }
I20251211 20:55:33.674927 7272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 43355 } } }
I20251211 20:55:33.674935 7273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b3dc1110bfe4a2fb24e89fe2c82fd16. Latest consensus state: current_term: 1 leader_uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1b3dc1110bfe4a2fb24e89fe2c82fd16" member_type: VOTER last_known_addr { host: "127.5.211.254" port: 43355 } } }
I20251211 20:55:33.675294 7272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [sys.catalog]: This master's current role is: LEADER
I20251211 20:55:33.675299 7273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16 [sys.catalog]: This master's current role is: LEADER
I20251211 20:55:33.677066 7281 catalog_manager.cc:1485] Loading table and tablet metadata into memory...
I20251211 20:55:33.683027 7281 catalog_manager.cc:1494] Initializing Kudu cluster ID...
I20251211 20:55:33.691068 7281 catalog_manager.cc:1357] Generated new cluster ID: 614e6edf0ef446b99dcef97d1b882c4b
I20251211 20:55:33.691180 7281 catalog_manager.cc:1505] Initializing Kudu internal certificate authority...
I20251211 20:55:33.709774 7281 catalog_manager.cc:1380] Generated new certificate authority record
I20251211 20:55:33.710907 7281 catalog_manager.cc:1514] Loading token signing keys...
I20251211 20:55:33.723145 7281 catalog_manager.cc:6027] T 00000000000000000000000000000000 P 1b3dc1110bfe4a2fb24e89fe2c82fd16: Generated new TSK 0
I20251211 20:55:33.723872 7281 catalog_manager.cc:1524] Initializing in-progress tserver states...
I20251211 20:55:33.729393 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.193:0
--local_ip_for_outbound_sockets=127.5.211.193
--webserver_interface=127.5.211.193
--webserver_port=0
--tserver_master_addrs=127.5.211.254:43355
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20251211 20:55:34.046355 7290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:34.046627 7290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:34.046738 7290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:34.054206 7290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:34.054366 7290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.193
I20251211 20:55:34.063093 7290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.211.193
--webserver_port=0
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.193
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:34.064533 7290 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:34.066321 7290 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:34.074927 7296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:34.075042 7295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:34.075596 7298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:34.076018 7290 server_base.cc:1047] running on GCE node
I20251211 20:55:34.076531 7290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:34.077332 7290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:34.078562 7290 hybrid_clock.cc:648] HybridClock initialized: now 1765486534078516 us; error 32 us; skew 500 ppm
I20251211 20:55:34.078855 7290 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:34.080915 7290 webserver.cc:492] Webserver started at http://127.5.211.193:41527/ using document root <none> and password file <none>
I20251211 20:55:34.081619 7290 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:34.081727 7290 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:34.082103 7290 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:55:34.084206 7290 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/instance:
uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:34.084952 7290 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal/instance:
uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:34.089748 7290 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.000s sys 0.007s
I20251211 20:55:34.092722 7304 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:34.093881 7290 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20251211 20:55:34.094048 7290 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:34.094230 7290 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:34.125308 7290 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:34.126147 7290 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:34.126375 7290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:34.127223 7290 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:34.128712 7290 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:55:34.128810 7290 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:34.128923 7290 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:55:34.128998 7290 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:34.159808 7290 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.193:37733
I20251211 20:55:34.159934 7416 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.193:37733 every 8 connection(s)
I20251211 20:55:34.161053 7290 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
I20251211 20:55:34.167726 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 7290
I20251211 20:55:34.167938 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal/instance
I20251211 20:55:34.170930 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.194:0
--local_ip_for_outbound_sockets=127.5.211.194
--webserver_interface=127.5.211.194
--webserver_port=0
--tserver_master_addrs=127.5.211.254:43355
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20251211 20:55:34.173036 7417 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:34.173362 7417 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:34.174266 7417 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:34.220633 7231 ts_manager.cc:194] Registered new tserver with Master: 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:55:34.222409 7231 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.193:33275
W20251211 20:55:34.486950 7421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:34.487260 7421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:34.487349 7421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:34.494758 7421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:34.494917 7421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.194
I20251211 20:55:34.503969 7421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.211.194
--webserver_port=0
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.194
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:34.505477 7421 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:34.507081 7421 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:34.516084 7427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:34.516180 7430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:34.516253 7428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:34.516788 7421 server_base.cc:1047] running on GCE node
I20251211 20:55:34.517318 7421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:34.518209 7421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:34.519397 7421 hybrid_clock.cc:648] HybridClock initialized: now 1765486534519364 us; error 26 us; skew 500 ppm
I20251211 20:55:34.519658 7421 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:34.521509 7421 webserver.cc:492] Webserver started at http://127.5.211.194:45957/ using document root <none> and password file <none>
I20251211 20:55:34.522219 7421 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:34.522341 7421 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:34.522660 7421 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:55:34.524665 7421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/instance:
uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:34.525367 7421 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal/instance:
uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:34.530257 7421 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20251211 20:55:34.533282 7436 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:34.534386 7421 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20251211 20:55:34.534530 7421 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:34.534703 7421 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:34.563491 7421 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:34.564168 7421 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:34.564416 7421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:34.565264 7421 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:34.566713 7421 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:55:34.566807 7421 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:34.566908 7421 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:55:34.566984 7421 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:34.598721 7421 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.194:45381
I20251211 20:55:34.598774 7548 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.194:45381 every 8 connection(s)
I20251211 20:55:34.599946 7421 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
I20251211 20:55:34.606580 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 7421
I20251211 20:55:34.606732 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal/instance
I20251211 20:55:34.609963 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.195:0
--local_ip_for_outbound_sockets=127.5.211.195
--webserver_interface=127.5.211.195
--webserver_port=0
--tserver_master_addrs=127.5.211.254:43355
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20251211 20:55:34.612025 7549 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:34.612329 7549 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:34.613155 7549 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:34.661862 7231 ts_manager.cc:194] Registered new tserver with Master: 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381)
I20251211 20:55:34.663249 7231 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.194:54145
W20251211 20:55:34.927562 7552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:34.927827 7552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:34.927902 7552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:34.935319 7552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:34.935461 7552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.195
I20251211 20:55:34.944170 7552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.211.195
--webserver_port=0
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.195
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:34.945604 7552 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:34.947319 7552 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:34.956349 7558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:34.956404 7559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:34.956385 7561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:34.957161 7552 server_base.cc:1047] running on GCE node
I20251211 20:55:34.957684 7552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:34.958500 7552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:34.959690 7552 hybrid_clock.cc:648] HybridClock initialized: now 1765486534959644 us; error 33 us; skew 500 ppm
I20251211 20:55:34.959949 7552 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:34.961846 7552 webserver.cc:492] Webserver started at http://127.5.211.195:37277/ using document root <none> and password file <none>
I20251211 20:55:34.962541 7552 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:34.962644 7552 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:34.962940 7552 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:55:34.965087 7552 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/instance:
uuid: "671b0c18e41544439fd978aa70f7ff53"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:34.965807 7552 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal/instance:
uuid: "671b0c18e41544439fd978aa70f7ff53"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:34.970763 7552 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20251211 20:55:34.973798 7567 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:34.975025 7552 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20251211 20:55:34.975171 7552 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
uuid: "671b0c18e41544439fd978aa70f7ff53"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:34.975353 7552 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:34.989578 7552 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:34.990408 7552 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:34.990669 7552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:34.991627 7552 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:34.993002 7552 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:55:34.993081 7552 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:34.993149 7552 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:55:34.993198 7552 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:35.024128 7552 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.195:37875
I20251211 20:55:35.024179 7679 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.195:37875 every 8 connection(s)
I20251211 20:55:35.025425 7552 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
I20251211 20:55:35.035612 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 7552
I20251211 20:55:35.035802 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal/instance
I20251211 20:55:35.038262 7680 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:35.038622 7680 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:35.039583 7680 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:35.040009 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.196:0
--local_ip_for_outbound_sockets=127.5.211.196
--webserver_interface=127.5.211.196
--webserver_port=0
--tserver_master_addrs=127.5.211.254:43355
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20251211 20:55:35.081257 7230 ts_manager.cc:194] Registered new tserver with Master: 671b0c18e41544439fd978aa70f7ff53 (127.5.211.195:37875)
I20251211 20:55:35.082255 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.195:59283
I20251211 20:55:35.226392 7417 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
W20251211 20:55:35.367661 7684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:35.367954 7684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:35.368077 7684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:35.375816 7684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:35.376008 7684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.196
I20251211 20:55:35.385015 7684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.196:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.5.211.196
--webserver_port=0
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.196
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:35.386551 7684 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:35.388373 7684 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:35.396682 7689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:35.397099 7692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:35.397363 7690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:35.397891 7684 server_base.cc:1047] running on GCE node
I20251211 20:55:35.398392 7684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:35.399186 7684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:35.400384 7684 hybrid_clock.cc:648] HybridClock initialized: now 1765486535400346 us; error 33 us; skew 500 ppm
I20251211 20:55:35.400663 7684 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:35.402751 7684 webserver.cc:492] Webserver started at http://127.5.211.196:39639/ using document root <none> and password file <none>
I20251211 20:55:35.403409 7684 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:35.403522 7684 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:35.403831 7684 server_base.cc:895] This appears to be a new deployment of Kudu; creating new FS layout
I20251211 20:55:35.405970 7684 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/instance:
uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
format_stamp: "Formatted at 2025-12-11 20:55:35 on dist-test-slave-sk6m"
I20251211 20:55:35.406677 7684 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal/instance:
uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
format_stamp: "Formatted at 2025-12-11 20:55:35 on dist-test-slave-sk6m"
I20251211 20:55:35.411396 7684 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20251211 20:55:35.414350 7698 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:35.415467 7684 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20251211 20:55:35.415609 7684 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
format_stamp: "Formatted at 2025-12-11 20:55:35 on dist-test-slave-sk6m"
I20251211 20:55:35.415784 7684 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:35.432121 7684 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:35.432920 7684 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:35.433172 7684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:35.434182 7684 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:35.435600 7684 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:55:35.435700 7684 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:35.435803 7684 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:55:35.435879 7684 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:35.466845 7684 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.196:35931
I20251211 20:55:35.466893 7810 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.196:35931 every 8 connection(s)
I20251211 20:55:35.468148 7684 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
I20251211 20:55:35.473224 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 7684
I20251211 20:55:35.473400 5967 external_mini_cluster.cc:1442] Reading /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal/instance
I20251211 20:55:35.483711 7811 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:35.483982 7811 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:35.484643 7811 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:35.526005 7230 ts_manager.cc:194] Registered new tserver with Master: 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931)
I20251211 20:55:35.526866 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.196:52731
I20251211 20:55:35.529904 5967 external_mini_cluster.cc:949] 4 TS(s) registered with all masters
I20251211 20:55:35.544359 5967 test_util.cc:276] Using random seed: 1573817863
I20251211 20:55:35.559052 7230 catalog_manager.cc:2257] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33682:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20251211 20:55:35.587637 7484 tablet_service.cc:1505] Processing CreateTablet for tablet fb7cb7e3685a4401a6d872d961a7fb17 (DEFAULT_TABLE table=test-workload [id=97ee1dba7a2a4093a1bda980bce92cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:55:35.589278 7484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb7cb7e3685a4401a6d872d961a7fb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:35.589140 7746 tablet_service.cc:1505] Processing CreateTablet for tablet fb7cb7e3685a4401a6d872d961a7fb17 (DEFAULT_TABLE table=test-workload [id=97ee1dba7a2a4093a1bda980bce92cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:55:35.590559 7746 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb7cb7e3685a4401a6d872d961a7fb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:35.590647 7352 tablet_service.cc:1505] Processing CreateTablet for tablet fb7cb7e3685a4401a6d872d961a7fb17 (DEFAULT_TABLE table=test-workload [id=97ee1dba7a2a4093a1bda980bce92cbd]), partition=RANGE (key) PARTITION UNBOUNDED
I20251211 20:55:35.591758 7352 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fb7cb7e3685a4401a6d872d961a7fb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20251211 20:55:35.608340 7835 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap starting.
I20251211 20:55:35.614452 7836 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap starting.
I20251211 20:55:35.617345 7836 tablet_bootstrap.cc:654] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Neither blocks nor log segments found. Creating new log.
I20251211 20:55:35.618155 7837 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap starting.
I20251211 20:55:35.620549 7835 tablet_bootstrap.cc:654] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Neither blocks nor log segments found. Creating new log.
I20251211 20:55:35.622356 7836 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Log is configured to *not* fsync() on all Append() calls
I20251211 20:55:35.622927 7837 tablet_bootstrap.cc:654] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Neither blocks nor log segments found. Creating new log.
I20251211 20:55:35.623143 7835 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Log is configured to *not* fsync() on all Append() calls
I20251211 20:55:35.624410 7837 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Log is configured to *not* fsync() on all Append() calls
I20251211 20:55:35.625744 7836 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: No bootstrap required, opened a new log
I20251211 20:55:35.626061 7836 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Time spent bootstrapping tablet: real 0.012s user 0.003s sys 0.003s
I20251211 20:55:35.626539 7835 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: No bootstrap required, opened a new log
I20251211 20:55:35.626775 7835 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.001s
I20251211 20:55:35.626904 7837 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: No bootstrap required, opened a new log
I20251211 20:55:35.627187 7837 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20251211 20:55:35.632515 7837 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:35.632817 7837 raft_consensus.cc:385] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:55:35.632918 7837 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c15ab665fb453fbea8cda8fd8e13d0, State: Initialized, Role: FOLLOWER
I20251211 20:55:35.633467 7837 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:35.633426 7835 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:35.633514 7836 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:35.633829 7835 raft_consensus.cc:385] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:55:35.633925 7835 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Initialized, Role: FOLLOWER
I20251211 20:55:35.634665 7835 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:35.634836 7836 raft_consensus.cc:385] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20251211 20:55:35.634948 7836 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d21816e6f984a25a7eb64e5c8b29a2c, State: Initialized, Role: FOLLOWER
I20251211 20:55:35.635721 7549 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:35.635641 7836 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:35.636344 7835 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Time spent starting tablet: real 0.009s user 0.005s sys 0.005s
I20251211 20:55:35.637281 7811 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:35.637822 7836 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Time spent starting tablet: real 0.012s user 0.001s sys 0.008s
I20251211 20:55:35.637954 7837 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Time spent starting tablet: real 0.011s user 0.005s sys 0.003s
W20251211 20:55:35.663426 7418 tablet.cc:2378] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:55:35.689805 7842 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:35.690038 7842 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:35.691573 7842 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:55:35.699949 7372 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" is_pre_election: true
I20251211 20:55:35.700356 7372 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 0.
I20251211 20:55:35.700368 7766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" is_pre_election: true
I20251211 20:55:35.700827 7766 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 0.
I20251211 20:55:35.700884 7439 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0, 5ccd26b0e8fe409f91fbd1da24bc7137; no voters:
I20251211 20:55:35.701270 7842 raft_consensus.cc:2804] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20251211 20:55:35.701448 7842 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:55:35.701568 7842 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:55:35.703722 7842 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:35.704464 7842 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 1 election: Requested vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:55:35.704850 7766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
I20251211 20:55:35.705015 7372 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
I20251211 20:55:35.705042 7766 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:55:35.705190 7372 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 0 FOLLOWER]: Advancing to term 1
I20251211 20:55:35.707090 7766 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 1.
I20251211 20:55:35.707201 7372 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 1.
I20251211 20:55:35.707494 7438 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ccd26b0e8fe409f91fbd1da24bc7137, 6d21816e6f984a25a7eb64e5c8b29a2c; no voters:
I20251211 20:55:35.707751 7842 raft_consensus.cc:2804] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 1 FOLLOWER]: Leader election won for term 1
I20251211 20:55:35.708148 7842 raft_consensus.cc:697] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 1 LEADER]: Becoming Leader. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Running, Role: LEADER
I20251211 20:55:35.708578 7842 consensus_queue.cc:237] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:35.712749 7230 catalog_manager.cc:5654] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194). New cstate: current_term: 1 leader_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } health_report { overall_health: UNKNOWN } } }
W20251211 20:55:35.719622 7812 tablet.cc:2378] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20251211 20:55:35.737001 5967 maintenance_mode-itest.cc:745] Restarting batch of 4 tservers: 5ccd26b0e8fe409f91fbd1da24bc7137,30c15ab665fb453fbea8cda8fd8e13d0,6d21816e6f984a25a7eb64e5c8b29a2c,671b0c18e41544439fd978aa70f7ff53
I20251211 20:55:35.815811 7842 consensus_queue.cc:1048] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20251211 20:55:35.834045 7842 consensus_queue.cc:1048] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20251211 20:55:35.852126 7550 tablet.cc:2378] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20251211 20:55:35.935739 7439 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.5.211.193:37733, user_credentials={real_user=slave}} blocked reactor thread for 57149.1us
I20251211 20:55:36.085510 7680 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:36.592037 7230 ts_manager.cc:295] Set tserver state for 671b0c18e41544439fd978aa70f7ff53 to MAINTENANCE_MODE
I20251211 20:55:36.939424 7230 ts_manager.cc:295] Set tserver state for 6d21816e6f984a25a7eb64e5c8b29a2c to MAINTENANCE_MODE
I20251211 20:55:37.014247 7226 ts_manager.cc:295] Set tserver state for 5ccd26b0e8fe409f91fbd1da24bc7137 to MAINTENANCE_MODE
I20251211 20:55:38.249500 7226 ts_manager.cc:295] Set tserver state for 30c15ab665fb453fbea8cda8fd8e13d0 to MAINTENANCE_MODE
I20251211 20:55:38.526582 7615 tablet_service.cc:1460] Tablet server 671b0c18e41544439fd978aa70f7ff53 set to quiescing
I20251211 20:55:38.526820 7615 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:55:38.775153 7746 tablet_service.cc:1460] Tablet server 6d21816e6f984a25a7eb64e5c8b29a2c set to quiescing
I20251211 20:55:38.775430 7746 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:55:39.419456 7484 tablet_service.cc:1460] Tablet server 5ccd26b0e8fe409f91fbd1da24bc7137 set to quiescing
I20251211 20:55:39.419742 7484 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251211 20:55:39.427039 7972 raft_consensus.cc:993] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: : Instructing follower 30c15ab665fb453fbea8cda8fd8e13d0 to start an election
I20251211 20:55:39.427260 7972 raft_consensus.cc:1081] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 1 LEADER]: Signalling peer 30c15ab665fb453fbea8cda8fd8e13d0 to start an election
I20251211 20:55:39.430822 7371 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17"
dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
from {username='slave'} at 127.5.211.194:50681
I20251211 20:55:39.431062 7371 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20251211 20:55:39.431182 7371 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 1 FOLLOWER]: Advancing to term 2
I20251211 20:55:39.433904 7371 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:39.441404 7371 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [CANDIDATE]: Term 2 election: Requested vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381)
I20251211 20:55:39.451365 7372 raft_consensus.cc:1240] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Rejecting Update request from peer 5ccd26b0e8fe409f91fbd1da24bc7137 for earlier term 1. Current term is 2. Ops: []
I20251211 20:55:39.467195 7870 consensus_queue.cc:1059] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 }, Status: INVALID_TERM, Last received: 1.237, Next index: 238, Last known committed idx: 236, Time since last communication: 0.000s
I20251211 20:55:39.467528 7870 raft_consensus.cc:3055] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 1 LEADER]: Stepping down as leader of term 1
I20251211 20:55:39.467634 7870 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Running, Role: LEADER
I20251211 20:55:39.467957 7870 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 241, Committed index: 241, Last appended: 1.241, Last appended by leader: 241, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:39.468467 7870 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 1 FOLLOWER]: Advancing to term 2
W20251211 20:55:39.471014 7462 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:39.480877 7766 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 237 } } ignore_live_leader: true dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
I20251211 20:55:39.481135 7766 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 1 FOLLOWER]: Advancing to term 2
I20251211 20:55:39.483952 7766 raft_consensus.cc:2410] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 30c15ab665fb453fbea8cda8fd8e13d0 for term 2 because replica has last-logged OpId of term: 1 index: 241, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 237.
I20251211 20:55:39.484441 7504 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" candidate_term: 2 candidate_status { last_received { term: 1 index: 237 } } ignore_live_leader: true dest_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
I20251211 20:55:39.484867 7504 raft_consensus.cc:2410] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 30c15ab665fb453fbea8cda8fd8e13d0 for term 2 because replica has last-logged OpId of term: 1 index: 241, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 237.
I20251211 20:55:39.485675 7306 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0; no voters: 5ccd26b0e8fe409f91fbd1da24bc7137, 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:55:39.490648 7981 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20251211 20:55:39.521972 7726 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.523383 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.524245 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.530526 7327 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.532088 7327 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.534133 7331 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.538156 7462 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.538158 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.539408 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.544509 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.547981 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.548532 7462 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.556593 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.557096 7331 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.557729 7330 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.567234 7462 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.567581 7332 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.569487 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.577189 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.582396 7462 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.582813 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.598517 7331 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.599478 7332 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.600227 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.612823 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.619047 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.623466 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.630585 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.630731 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.638545 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.643956 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.648311 7330 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.652155 7330 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.660437 7331 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.666182 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.667522 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:39.673071 7352 tablet_service.cc:1460] Tablet server 30c15ab665fb453fbea8cda8fd8e13d0 set to quiescing
I20251211 20:55:39.673316 7352 tablet_service.cc:1467] Tablet server has 0 leaders and 3 scanners
W20251211 20:55:39.677860 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.683462 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.685595 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.696144 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.700651 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.703927 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.715349 7332 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.719704 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.723109 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.734944 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.742093 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.742093 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.755155 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.762185 7330 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.762202 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.764086 7995 raft_consensus.cc:670] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: failed to trigger leader election: Illegal state: leader elections are disabled
W20251211 20:55:39.778050 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.785108 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.786917 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.802796 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.808276 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.809701 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.825435 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.826983 7870 raft_consensus.cc:670] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: failed to trigger leader election: Illegal state: leader elections are disabled
W20251211 20:55:39.832883 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.834352 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.847741 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.854771 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.862761 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.873255 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.878700 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.889132 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.899893 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.904137 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.914814 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.924448 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.930524 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.935254 7981 raft_consensus.cc:670] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: failed to trigger leader election: Illegal state: leader elections are disabled
W20251211 20:55:39.945039 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.951381 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.958657 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:39.973033 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.981308 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:39.989534 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.005007 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.010139 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.020290 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.037199 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.043738 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.052374 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.068040 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.074710 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.084225 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.100064 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.110146 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.117271 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.133011 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.145494 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.151973 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.168514 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.181123 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.188601 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.202214 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.215337 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.226617 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.240960 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.251499 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.263988 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.276592 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.291173 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.303345 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.317066 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.328083 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.342428 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.358626 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.369092 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.384651 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.400074 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.411425 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.424834 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.440704 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.454984 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.469192 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.487144 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.501513 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.513697 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.530083 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.544466 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.557808 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.577306 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.587448 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.601612 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.622622 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.637645 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.650084 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.668877 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.685654 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.699316 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.718986 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.732208 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.745626 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.771396 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.784185 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.794293 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.819988 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.835304 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.846877 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:40.872443 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.889374 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.900936 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.926131 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.942400 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.955291 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:40.978266 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:40.987700 7484 tablet_service.cc:1460] Tablet server 5ccd26b0e8fe409f91fbd1da24bc7137 set to quiescing
I20251211 20:55:40.988039 7484 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251211 20:55:40.995123 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.009724 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.031477 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.051826 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.062902 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.090761 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.109879 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.118292 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.147722 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.164570 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.176322 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:41.187475 7352 tablet_service.cc:1460] Tablet server 30c15ab665fb453fbea8cda8fd8e13d0 set to quiescing
I20251211 20:55:41.187641 7352 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251211 20:55:41.203187 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.220531 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.235656 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.262145 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.280810 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.297677 7463 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:46160: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:41.319278 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 7421
W20251211 20:55:41.320647 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:41.325675 7544 generic_service.cc:196] Checking for leaks (request via RPC)
W20251211 20:55:41.339511 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.358309 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.383157 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.399344 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.419785 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:55:41.431301 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 7421
W20251211 20:55:41.452598 7821 connection.cc:537] client connection to 127.5.211.194:45381 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:55:41.453001 7821 meta_cache.cc:302] tablet fb7cb7e3685a4401a6d872d961a7fb17: replica 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381) has failed: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:55:41.453524 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.194:45381
--local_ip_for_outbound_sockets=127.5.211.194
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=45957
--webserver_interface=127.5.211.194
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20251211 20:55:41.515074 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.525877 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.543754 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.578187 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.589479 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.609656 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.708695 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.716362 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.743134 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:41.774390 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.780746 8022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:41.781038 8022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:41.781167 8022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:41.783479 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.789485 8022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:41.789680 8022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.194
I20251211 20:55:41.799039 8022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.194:45381
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.211.194
--webserver_port=45957
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.194
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:41.800550 8022 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:41.802366 8022 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:41.809134 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:41.811944 8028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:41.812171 8031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:41.812258 8029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:41.813618 8022 server_base.cc:1047] running on GCE node
I20251211 20:55:41.814201 8022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:41.815014 8022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:41.816205 8022 hybrid_clock.cc:648] HybridClock initialized: now 1765486541816161 us; error 34 us; skew 500 ppm
I20251211 20:55:41.816476 8022 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:41.818451 8022 webserver.cc:492] Webserver started at http://127.5.211.194:45957/ using document root <none> and password file <none>
I20251211 20:55:41.819079 8022 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:41.819190 8022 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:41.823777 8022 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20251211 20:55:41.826547 8037 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:41.827797 8022 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20251211 20:55:41.827941 8022 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:41.828577 8022 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:41.846230 8022 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:41.846982 8022 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:41.847251 8022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:41.848099 8022 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:41.850294 8044 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251211 20:55:41.856321 8022 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251211 20:55:41.856413 8022 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20251211 20:55:41.856535 8022 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251211 20:55:41.859340 8022 ts_tablet_manager.cc:616] Registered 1 tablets
I20251211 20:55:41.859439 8022 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s user 0.000s sys 0.002s
I20251211 20:55:41.859807 8044 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap starting.
I20251211 20:55:41.903199 8022 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.194:45381
I20251211 20:55:41.903240 8150 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.194:45381 every 8 connection(s)
I20251211 20:55:41.905068 8022 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
I20251211 20:55:41.909658 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 8022
I20251211 20:55:41.909894 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 7290
W20251211 20:55:41.911425 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:41.920601 8151 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:41.920984 8151 heartbeater.cc:461] Registering TS with master...
W20251211 20:55:41.921178 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:41.921242 7412 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:55:41.922050 8151 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:41.924893 7230 ts_manager.cc:194] Re-registered known tserver with Master: 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381)
I20251211 20:55:41.926167 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.194:48029
W20251211 20:55:41.945206 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:41.958402 8044 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Log is configured to *not* fsync() on all Append() calls
I20251211 20:55:42.064879 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 7290
W20251211 20:55:42.064842 7329 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:42.064901 7330 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55076: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:55:42.101343 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.193:37733
--local_ip_for_outbound_sockets=127.5.211.193
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=41527
--webserver_interface=127.5.211.193
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20251211 20:55:42.239059 7851 meta_cache.cc:1510] marking tablet server 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) as failed
W20251211 20:55:42.255596 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:42.256407 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:42.306849 7852 meta_cache.cc:1510] marking tablet server 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) as failed
W20251211 20:55:42.331681 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:42.431440 8159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:42.431777 8159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:42.431893 8159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:42.439774 8159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:42.439956 8159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.193
I20251211 20:55:42.449359 8159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.193:37733
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.211.193
--webserver_port=41527
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.193
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:42.450899 8159 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:42.452625 8159 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:42.461333 8166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:42.462218 8165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:42.463639 8168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:42.463897 8159 server_base.cc:1047] running on GCE node
I20251211 20:55:42.464423 8159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:42.465293 8159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:42.466533 8159 hybrid_clock.cc:648] HybridClock initialized: now 1765486542466475 us; error 40 us; skew 500 ppm
I20251211 20:55:42.466858 8159 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:42.469470 8159 webserver.cc:492] Webserver started at http://127.5.211.193:41527/ using document root <none> and password file <none>
I20251211 20:55:42.470345 8159 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:42.470463 8159 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:42.476828 8159 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20251211 20:55:42.480661 8174 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
W20251211 20:55:42.480890 7725 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:42.481338 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:42.482249 8159 fs_manager.cc:730] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20251211 20:55:42.482431 8159 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:42.483249 8159 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:42.513568 8159 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:42.514526 8159 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:42.514797 8159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:42.515799 8159 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:42.518146 8181 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251211 20:55:42.523416 8159 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251211 20:55:42.523507 8159 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20251211 20:55:42.523629 8159 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251211 20:55:42.527503 8159 ts_tablet_manager.cc:616] Registered 1 tablets
I20251211 20:55:42.527611 8159 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.000s sys 0.002s
I20251211 20:55:42.528048 8181 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap starting.
I20251211 20:55:42.540745 8044 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap replayed 1/1 log segments. Stats: ops{read=241 overwritten=0 applied=241 ignored=0} inserts{seen=12000 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:55:42.541538 8044 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap complete.
I20251211 20:55:42.543578 8044 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Time spent bootstrapping tablet: real 0.684s user 0.622s sys 0.058s
I20251211 20:55:42.550388 8044 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:42.551594 8044 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Initialized, Role: FOLLOWER
I20251211 20:55:42.552414 8044 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 241, Last appended: 1.241, Last appended by leader: 241, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:42.553820 8151 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:42.554380 8044 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Time spent starting tablet: real 0.011s user 0.008s sys 0.002s
W20251211 20:55:42.554469 7822 meta_cache.cc:302] tablet fb7cb7e3685a4401a6d872d961a7fb17: replica 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) has failed: Network error: Client connection negotiation failed: client connection to 127.5.211.193:37733: connect: Connection refused (error 111)
W20251211 20:55:42.556169 7724 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59256: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:42.560875 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:42.576084 8159 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.193:37733
I20251211 20:55:42.576102 8288 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.193:37733 every 8 connection(s)
I20251211 20:55:42.577836 8159 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
I20251211 20:55:42.587630 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 8159
I20251211 20:55:42.587832 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 7684
I20251211 20:55:42.593626 8289 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:42.594033 8289 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:42.595149 8289 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:42.595916 7806 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:55:42.598268 7226 ts_manager.cc:194] Re-registered known tserver with Master: 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:55:42.599563 7226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.193:58641
I20251211 20:55:42.631506 8181 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Log is configured to *not* fsync() on all Append() calls
I20251211 20:55:42.712633 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 7684
W20251211 20:55:42.721129 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:42.722585 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:42.739328 7821 connection.cc:537] client connection to 127.5.211.196:35931 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:55:42.740311 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.196:35931
--local_ip_for_outbound_sockets=127.5.211.196
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=39639
--webserver_interface=127.5.211.196
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20251211 20:55:42.818954 8231 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:42.819188 8231 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:42.820829 8231 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
W20251211 20:55:42.820909 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:42.822743 8039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.211.196:35931: connect: Connection refused (error 111)
W20251211 20:55:42.832201 8039 leader_election.cc:336] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Network error: Client connection negotiation failed: client connection to 127.5.211.196:35931: connect: Connection refused (error 111)
I20251211 20:55:42.832924 8244 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 3 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" is_pre_election: true
W20251211 20:55:42.835500 8040 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:55:42.835675 8040 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ccd26b0e8fe409f91fbd1da24bc7137; no voters: 30c15ab665fb453fbea8cda8fd8e13d0, 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:55:42.836050 8231 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20251211 20:55:42.882746 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:42.963342 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:43.047775 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:43.111696 8298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:43.111976 8298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:43.112110 8298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:43.119976 8298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:43.120158 8298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.196
W20251211 20:55:43.126910 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:43.130637 8298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.196:35931
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.5.211.196
--webserver_port=39639
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.196
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:43.132095 8298 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:43.133843 8298 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:43.142766 8307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:43.143488 8306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:43.144783 8309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:43.145123 8298 server_base.cc:1047] running on GCE node
I20251211 20:55:43.145644 8298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:43.146584 8298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:43.147811 8298 hybrid_clock.cc:648] HybridClock initialized: now 1765486543147778 us; error 26 us; skew 500 ppm
I20251211 20:55:43.148082 8298 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:43.150157 8298 webserver.cc:492] Webserver started at http://127.5.211.196:39639/ using document root <none> and password file <none>
I20251211 20:55:43.150825 8298 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:43.150921 8298 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:43.155820 8298 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20251211 20:55:43.158422 8315 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:43.159667 8298 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20251211 20:55:43.159790 8298 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
format_stamp: "Formatted at 2025-12-11 20:55:35 on dist-test-slave-sk6m"
I20251211 20:55:43.160400 8298 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:43.182449 8298 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:43.183131 8298 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:43.183408 8298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:43.184291 8298 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:43.186834 8322 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251211 20:55:43.192062 8298 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251211 20:55:43.192137 8298 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20251211 20:55:43.192234 8298 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251211 20:55:43.195302 8298 ts_tablet_manager.cc:616] Registered 1 tablets
I20251211 20:55:43.195421 8298 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20251211 20:55:43.195879 8322 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap starting.
W20251211 20:55:43.224843 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:43.227063 8231 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:43.227257 8231 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:43.227918 8231 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:55:43.229084 8244 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 3 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" is_pre_election: true
W20251211 20:55:43.229700 8040 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:55:43.230406 8039 leader_election.cc:336] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Network error: Client connection negotiation failed: client connection to 127.5.211.196:35931: connect: Connection refused (error 111)
I20251211 20:55:43.230517 8039 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ccd26b0e8fe409f91fbd1da24bc7137; no voters: 30c15ab665fb453fbea8cda8fd8e13d0, 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:55:43.230998 8231 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20251211 20:55:43.262348 8298 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.196:35931
I20251211 20:55:43.262401 8428 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.196:35931 every 8 connection(s)
I20251211 20:55:43.264442 8298 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
I20251211 20:55:43.268105 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 8298
I20251211 20:55:43.268379 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 7552
I20251211 20:55:43.281241 8181 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=237 overwritten=0 applied=236 ignored=0} inserts{seen=11750 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251211 20:55:43.282083 8181 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap complete.
I20251211 20:55:43.283665 8181 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Time spent bootstrapping tablet: real 0.756s user 0.707s sys 0.036s
I20251211 20:55:43.284334 7675 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:55:43.287092 8429 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:43.286759 8181 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:43.287590 8429 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:43.288928 8429 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:43.292938 7226 ts_manager.cc:194] Re-registered known tserver with Master: 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931)
I20251211 20:55:43.294921 7226 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.196:39955
I20251211 20:55:43.299712 8181 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c15ab665fb453fbea8cda8fd8e13d0, State: Initialized, Role: FOLLOWER
I20251211 20:55:43.300422 8181 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 236, Last appended: 1.237, Last appended by leader: 237, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:43.314193 8289 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:43.314873 8181 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Time spent starting tablet: real 0.031s user 0.007s sys 0.016s
I20251211 20:55:43.336380 8322 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Log is configured to *not* fsync() on all Append() calls
I20251211 20:55:43.383436 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 7552
W20251211 20:55:43.397908 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:43.399137 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:43.400858 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:43.401605 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.195:37875
--local_ip_for_outbound_sockets=127.5.211.195
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=37277
--webserver_interface=127.5.211.195
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20251211 20:55:43.488569 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:43.540189 8438 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:43.540551 8438 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:43.542598 8438 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381)
I20251211 20:55:43.554988 8384 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" candidate_term: 3 candidate_status { last_received { term: 1 index: 237 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" is_pre_election: true
W20251211 20:55:43.558728 8176 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:55:43.558863 8106 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" candidate_term: 3 candidate_status { last_received { term: 1 index: 237 } } ignore_live_leader: false dest_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" is_pre_election: true
I20251211 20:55:43.559412 8106 raft_consensus.cc:2410] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 30c15ab665fb453fbea8cda8fd8e13d0 for term 3 because replica has last-logged OpId of term: 1 index: 241, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 237.
I20251211 20:55:43.560153 8176 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0; no voters: 5ccd26b0e8fe409f91fbd1da24bc7137, 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:55:43.561259 8438 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20251211 20:55:43.572613 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:43.573977 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:43.574265 8201 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:43.661760 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:43.665266 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:43.727506 8231 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:43.727843 8231 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:43.728601 8231 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:55:43.729360 8244 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 3 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" is_pre_election: true
I20251211 20:55:43.729648 8244 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 2.
I20251211 20:55:43.730228 8040 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0, 5ccd26b0e8fe409f91fbd1da24bc7137; no voters:
I20251211 20:55:43.730690 8231 raft_consensus.cc:2804] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20251211 20:55:43.730832 8231 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:55:43.730947 8231 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 2 FOLLOWER]: Advancing to term 3
I20251211 20:55:43.735826 8231 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:43.736685 8231 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 election: Requested vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:55:43.737648 8383 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 3 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
I20251211 20:55:43.737674 8384 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 3 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" is_pre_election: true
W20251211 20:55:43.738373 8039 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:55:43.738705 8244 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 3 candidate_status { last_received { term: 1 index: 241 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
W20251211 20:55:43.738821 8039 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:55:43.738942 8244 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 2 FOLLOWER]: Advancing to term 3
I20251211 20:55:43.743311 8244 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 3.
I20251211 20:55:43.743700 8040 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0, 5ccd26b0e8fe409f91fbd1da24bc7137; no voters: 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:55:43.744005 8231 raft_consensus.cc:2804] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 3 FOLLOWER]: Leader election won for term 3
I20251211 20:55:43.745132 8231 raft_consensus.cc:697] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 3 LEADER]: Becoming Leader. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Running, Role: LEADER
I20251211 20:55:43.745668 8231 consensus_queue.cc:237] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 241, Committed index: 241, Last appended: 1.241, Last appended by leader: 241, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
W20251211 20:55:43.750097 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Service unavailable: leader is not yet ready
W20251211 20:55:43.753115 8440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:43.753486 8440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:43.753614 8440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20251211 20:55:43.754853 7226 catalog_manager.cc:5654] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } health_report { overall_health: UNKNOWN } } }
W20251211 20:55:43.765695 8440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:43.765893 8440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.195
I20251211 20:55:43.775404 8440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.195:37875
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.211.195
--webserver_port=37277
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.195
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:43.776978 8440 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:43.778600 8440 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:43.787209 8455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:43.787295 8454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:43.791008 8457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:43.791997 8440 server_base.cc:1047] running on GCE node
I20251211 20:55:43.792642 8440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:43.793715 8440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:43.794940 8440 hybrid_clock.cc:648] HybridClock initialized: now 1765486543794875 us; error 52 us; skew 500 ppm
I20251211 20:55:43.795291 8440 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:43.797823 8440 webserver.cc:492] Webserver started at http://127.5.211.195:37277/ using document root <none> and password file <none>
I20251211 20:55:43.798758 8440 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:43.798892 8440 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:43.805214 8440 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20251211 20:55:43.808385 8463 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:43.809831 8440 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20251211 20:55:43.809978 8440 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
uuid: "671b0c18e41544439fd978aa70f7ff53"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:43.810909 8440 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:43.831200 8440 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:43.832197 8440 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:43.832522 8440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:43.834496 8440 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
W20251211 20:55:43.836220 7853 scanner-internal.cc:458] Time spent opening tablet: real 3.922s user 0.005s sys 0.001s
W20251211 20:55:43.848210 8039 consensus_peers.cc:597] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 -> Peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Couldn't send request to peer 6d21816e6f984a25a7eb64e5c8b29a2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20251211 20:55:43.850143 7851 scanner-internal.cc:458] Time spent opening tablet: real 4.024s user 0.002s sys 0.004s
W20251211 20:55:43.850162 8203 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:43.851483 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:55:43.853781 8244 raft_consensus.cc:1275] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 3 FOLLOWER]: Refusing update from remote peer 5ccd26b0e8fe409f91fbd1da24bc7137: Log matching property violated. Preceding OpId in replica: term: 1 index: 237. Preceding OpId from leader: term: 3 index: 242. (index mismatch)
I20251211 20:55:43.854588 8231 consensus_queue.cc:1048] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 242, Last known committed idx: 236, Time since last communication: 0.000s
W20251211 20:55:43.925838 7852 scanner-internal.cc:458] Time spent opening tablet: real 4.031s user 0.005s sys 0.000s
W20251211 20:55:43.934769 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:55:43.996754 8440 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:55:43.996950 8440 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20251211 20:55:43.997123 8440 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:55:43.997237 8440 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:44.022344 8474 mvcc.cc:204] Tried to move back new op lower bound from 7231432884207038464 to 7231432883186487296. Current Snapshot: MvccSnapshot[applied={T|T < 7231432884162949120}]
I20251211 20:55:44.084733 8322 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=241 overwritten=0 applied=241 ignored=0} inserts{seen=12000 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:55:44.085747 8322 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap complete.
I20251211 20:55:44.090884 8322 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Time spent bootstrapping tablet: real 0.895s user 0.736s sys 0.081s
I20251211 20:55:44.097384 8322 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:44.098616 8322 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d21816e6f984a25a7eb64e5c8b29a2c, State: Initialized, Role: FOLLOWER
I20251211 20:55:44.099650 8322 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 241, Last appended: 1.241, Last appended by leader: 241, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:44.108351 8322 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Time spent starting tablet: real 0.017s user 0.009s sys 0.006s
I20251211 20:55:44.108943 8429 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:44.148672 8384 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 2 FOLLOWER]: Advancing to term 3
I20251211 20:55:44.243441 8495 mvcc.cc:204] Tried to move back new op lower bound from 7231432884207038464 to 7231432883186487296. Current Snapshot: MvccSnapshot[applied={T|T < 7231432884207038464 or (T in {7231432884207038464})}]
I20251211 20:55:45.382748 8440 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.195:37875
I20251211 20:55:45.382819 8595 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.195:37875 every 8 connection(s)
I20251211 20:55:45.384987 8440 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
I20251211 20:55:45.395267 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 8440
I20251211 20:55:45.453095 8596 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:45.453509 8596 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:45.454631 8596 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:45.456213 7230 ts_manager.cc:194] Re-registered known tserver with Master: 671b0c18e41544439fd978aa70f7ff53 (127.5.211.195:37875)
I20251211 20:55:45.457317 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.195:48989
I20251211 20:55:46.460388 8596 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:46.558334 8223 tablet_service.cc:1467] Tablet server has 0 leaders and 3 scanners
I20251211 20:55:46.591291 8086 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251211 20:55:46.618379 8364 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:55:46.677003 8530 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:55:48.556010 7230 ts_manager.cc:284] Unset tserver state for 30c15ab665fb453fbea8cda8fd8e13d0 from MAINTENANCE_MODE
I20251211 20:55:48.736604 7230 ts_manager.cc:284] Unset tserver state for 671b0c18e41544439fd978aa70f7ff53 from MAINTENANCE_MODE
I20251211 20:55:48.745497 7230 ts_manager.cc:284] Unset tserver state for 5ccd26b0e8fe409f91fbd1da24bc7137 from MAINTENANCE_MODE
I20251211 20:55:48.963626 7230 ts_manager.cc:284] Unset tserver state for 6d21816e6f984a25a7eb64e5c8b29a2c from MAINTENANCE_MODE
I20251211 20:55:48.967170 8289 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:49.292922 8429 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:49.315490 8151 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:49.471251 8596 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:50.367120 7230 ts_manager.cc:295] Set tserver state for 5ccd26b0e8fe409f91fbd1da24bc7137 to MAINTENANCE_MODE
I20251211 20:55:50.559759 7226 ts_manager.cc:295] Set tserver state for 671b0c18e41544439fd978aa70f7ff53 to MAINTENANCE_MODE
I20251211 20:55:50.733825 7226 ts_manager.cc:295] Set tserver state for 6d21816e6f984a25a7eb64e5c8b29a2c to MAINTENANCE_MODE
I20251211 20:55:51.532467 7226 ts_manager.cc:295] Set tserver state for 30c15ab665fb453fbea8cda8fd8e13d0 to MAINTENANCE_MODE
I20251211 20:55:52.285020 8530 tablet_service.cc:1460] Tablet server 671b0c18e41544439fd978aa70f7ff53 set to quiescing
I20251211 20:55:52.285320 8530 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:55:52.697945 8364 tablet_service.cc:1460] Tablet server 6d21816e6f984a25a7eb64e5c8b29a2c set to quiescing
I20251211 20:55:52.698222 8364 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:55:52.823534 8086 tablet_service.cc:1460] Tablet server 5ccd26b0e8fe409f91fbd1da24bc7137 set to quiescing
I20251211 20:55:52.823798 8086 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251211 20:55:52.840206 8637 raft_consensus.cc:993] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: : Instructing follower 30c15ab665fb453fbea8cda8fd8e13d0 to start an election
I20251211 20:55:52.852941 8758 raft_consensus.cc:1081] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 3 LEADER]: Signalling peer 30c15ab665fb453fbea8cda8fd8e13d0 to start an election
I20251211 20:55:52.861510 8244 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17"
dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
from {username='slave'} at 127.5.211.194:44261
I20251211 20:55:52.862520 8244 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20251211 20:55:52.862766 8244 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 3 FOLLOWER]: Advancing to term 4
I20251211 20:55:52.884048 8244 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:52.895730 8243 raft_consensus.cc:1240] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 4 FOLLOWER]: Rejecting Update request from peer 5ccd26b0e8fe409f91fbd1da24bc7137 for earlier term 3. Current term is 4. Ops: [3.899-3.899]
I20251211 20:55:52.896665 8637 consensus_queue.cc:1059] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 }, Status: INVALID_TERM, Last received: 3.898, Next index: 899, Last known committed idx: 898, Time since last communication: 0.000s
I20251211 20:55:52.900766 8383 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 898 } } ignore_live_leader: true dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
I20251211 20:55:52.901074 8383 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 3 FOLLOWER]: Advancing to term 4
I20251211 20:55:52.904354 8383 raft_consensus.cc:2410] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 30c15ab665fb453fbea8cda8fd8e13d0 for term 4 because replica has last-logged OpId of term: 3 index: 900, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 898.
I20251211 20:55:52.918145 8106 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" candidate_term: 4 candidate_status { last_received { term: 3 index: 898 } } ignore_live_leader: true dest_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
I20251211 20:55:52.917639 8244 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [CANDIDATE]: Term 4 election: Requested vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381)
I20251211 20:55:52.928964 8758 raft_consensus.cc:3055] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 3 LEADER]: Stepping down as leader of term 3
I20251211 20:55:52.929101 8758 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Running, Role: LEADER
I20251211 20:55:52.929442 8758 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 900, Committed index: 900, Last appended: 3.901, Last appended by leader: 901, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:52.930033 8758 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 3 FOLLOWER]: Advancing to term 4
W20251211 20:55:52.935070 8474 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.040s ago.
I20251211 20:55:52.936062 8176 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0; no voters: 5ccd26b0e8fe409f91fbd1da24bc7137, 6d21816e6f984a25a7eb64e5c8b29a2c
W20251211 20:55:52.941210 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:52.943403 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:52.949235 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:52.953853 8808 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20251211 20:55:52.956385 8201 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:52.956347 8198 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:52.975466 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:52.975831 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:52.986083 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:52.987030 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:52.994185 8200 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:52.994940 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.006765 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.010440 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.019119 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.020759 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.032761 8200 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.033579 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.048533 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.050446 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.064239 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.065631 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.078285 8200 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.078285 8203 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.096109 8064 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.096465 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.114408 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.114423 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.132428 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.134887 8198 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.153872 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.155375 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.173115 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.176873 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.194378 8812 raft_consensus.cc:670] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: failed to trigger leader election: Illegal state: leader elections are disabled
W20251211 20:55:53.195756 8203 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.197500 8198 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.216902 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.219905 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:53.238888 8223 tablet_service.cc:1460] Tablet server 30c15ab665fb453fbea8cda8fd8e13d0 set to quiescing
I20251211 20:55:53.239053 8223 tablet_service.cc:1467] Tablet server has 0 leaders and 2 scanners
W20251211 20:55:53.239823 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.241523 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.263758 8198 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.264561 8198 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.290661 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.297017 8808 raft_consensus.cc:670] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: failed to trigger leader election: Illegal state: leader elections are disabled
W20251211 20:55:53.297747 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.299510 8758 raft_consensus.cc:670] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: failed to trigger leader election: Illegal state: leader elections are disabled
W20251211 20:55:53.314476 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.324047 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.343606 8198 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.349754 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.372020 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.381727 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.400954 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.411666 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.431504 8203 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.443504 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.465452 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.474689 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.496374 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.505666 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.526975 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.539162 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.560263 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.576853 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.595502 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.613962 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.631460 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.650894 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.671022 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.690881 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.708361 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.726892 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.745420 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.763887 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.787797 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.806497 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.831184 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.849123 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.871745 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.892976 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.915027 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.936805 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:53.957533 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:53.981151 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.001788 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.028069 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.049284 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.074936 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.095878 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.123014 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.141901 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.170235 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.193897 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.220021 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.242498 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.271978 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.293861 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.321412 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.347530 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.373878 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.402752 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.429073 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:54.445531 8086 tablet_service.cc:1460] Tablet server 5ccd26b0e8fe409f91fbd1da24bc7137 set to quiescing
I20251211 20:55:54.445765 8086 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251211 20:55:54.468430 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.481921 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.528383 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.537840 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.586935 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.595671 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.643774 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.652801 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.702298 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.712046 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.760066 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:54.768034 8223 tablet_service.cc:1460] Tablet server 30c15ab665fb453fbea8cda8fd8e13d0 set to quiescing
I20251211 20:55:54.768230 8223 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251211 20:55:54.770732 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.818503 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.829728 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.880404 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:54.892557 8066 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:55920: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:55:54.900755 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 8022
I20251211 20:55:54.906781 8146 generic_service.cc:196] Checking for leaks (request via RPC)
W20251211 20:55:54.943347 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:54.955112 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.006799 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.017000 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:55:55.051667 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 8022
W20251211 20:55:55.078110 7821 connection.cc:537] client connection to 127.5.211.194:45381 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:55:55.078325 7821 meta_cache.cc:302] tablet fb7cb7e3685a4401a6d872d961a7fb17: replica 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381) has failed: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:55:55.078811 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.194:45381
--local_ip_for_outbound_sockets=127.5.211.194
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=45957
--webserver_interface=127.5.211.194
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20251211 20:55:55.092327 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.092525 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.096218 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.113595 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.123235 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.145136 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.147591 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.157091 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.158200 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.182926 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.195498 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.210053 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.227017 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.246639 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.286854 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.294847 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.307618 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.345323 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.354039 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.364782 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.377941 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.412298 8847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:55.412606 8847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:55.412716 8847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:55.413277 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:55:55.420676 8847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:55.420878 8847 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.194
W20251211 20:55:55.429663 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:55:55.430274 8847 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.194:45381
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.211.194
--webserver_port=45957
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.194
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:55.431838 8847 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:55.433813 8847 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:55.442694 8854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:55.442759 8853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:55.443313 8856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:55.443679 8847 server_base.cc:1047] running on GCE node
I20251211 20:55:55.444193 8847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:55.445053 8847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:55.446250 8847 hybrid_clock.cc:648] HybridClock initialized: now 1765486555446193 us; error 46 us; skew 500 ppm
I20251211 20:55:55.446523 8847 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:55.448616 8847 webserver.cc:492] Webserver started at http://127.5.211.194:45957/ using document root <none> and password file <none>
I20251211 20:55:55.449267 8847 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:55.449376 8847 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:55.454051 8847 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
W20251211 20:55:55.455621 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:55.457024 8862 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:55.458264 8847 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20251211 20:55:55.458406 8847 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:55.459134 8847 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:55.505471 8847 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
W20251211 20:55:55.506261 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:55.506651 8847 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:55.507056 8847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:55.508329 8847 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:55.510815 8869 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
W20251211 20:55:55.514823 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:55.516391 8847 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251211 20:55:55.516469 8847 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20251211 20:55:55.516583 8847 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251211 20:55:55.519526 8847 ts_tablet_manager.cc:616] Registered 1 tablets
I20251211 20:55:55.519603 8847 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20251211 20:55:55.520103 8869 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap starting.
W20251211 20:55:55.544633 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:55:55.565548 8847 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.194:45381
I20251211 20:55:55.565632 8975 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.194:45381 every 8 connection(s)
I20251211 20:55:55.566998 8847 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
I20251211 20:55:55.573079 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 8847
I20251211 20:55:55.573290 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 8159
W20251211 20:55:55.578581 8202 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:55:55.588850 8976 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:55.589164 8976 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:55.590446 8976 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:55.593499 7230 ts_manager.cc:194] Re-registered known tserver with Master: 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381)
I20251211 20:55:55.594913 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.194:42939
I20251211 20:55:55.602239 8283 generic_service.cc:196] Checking for leaks (request via RPC)
W20251211 20:55:55.631928 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.633069 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:55.780083 8198 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39862: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:55:55.780804 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 8159
I20251211 20:55:55.786182 8869 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Log is configured to *not* fsync() on all Append() calls
W20251211 20:55:55.805068 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:55.829473 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.193:37733
--local_ip_for_outbound_sockets=127.5.211.193
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=41527
--webserver_interface=127.5.211.193
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20251211 20:55:55.901894 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:56.006419 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:56.030560 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:56.037948 7852 meta_cache.cc:1510] marking tablet server 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) as failed
W20251211 20:55:56.052577 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:56.094825 7822 meta_cache.cc:302] tablet fb7cb7e3685a4401a6d872d961a7fb17: replica 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) has failed: Network error: Client connection negotiation failed: client connection to 127.5.211.193:37733: connect: Connection refused (error 111)
W20251211 20:55:56.160813 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:56.191262 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:56.201814 8985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:56.202093 8985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:56.202203 8985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:56.210083 8985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:56.210255 8985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.193
I20251211 20:55:56.219233 8985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.193:37733
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.211.193
--webserver_port=41527
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.193
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:56.220983 8985 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
W20251211 20:55:56.221191 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:56.222848 8985 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:56.231809 8990 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:56.231838 8991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:56.233219 8993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:56.233577 8985 server_base.cc:1047] running on GCE node
I20251211 20:55:56.234158 8985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:56.234962 8985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:56.236143 8985 hybrid_clock.cc:648] HybridClock initialized: now 1765486556236090 us; error 51 us; skew 500 ppm
I20251211 20:55:56.236402 8985 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:56.238538 8985 webserver.cc:492] Webserver started at http://127.5.211.193:41527/ using document root <none> and password file <none>
I20251211 20:55:56.239250 8985 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:56.239369 8985 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:56.244376 8985 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20251211 20:55:56.247143 8999 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:56.248335 8985 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20251211 20:55:56.248471 8985 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:56.249076 8985 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:56.268647 8985 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:56.269603 8985 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:56.269927 8985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:56.271044 8985 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:56.273531 9006 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251211 20:55:56.279172 8985 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251211 20:55:56.279287 8985 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20251211 20:55:56.279425 8985 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251211 20:55:56.283340 8985 ts_tablet_manager.cc:616] Registered 1 tablets
I20251211 20:55:56.283473 8985 ts_tablet_manager.cc:595] Time spent register tablets: real 0.004s user 0.003s sys 0.000s
I20251211 20:55:56.283879 9006 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap starting.
W20251211 20:55:56.309029 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:56.358784 8985 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.193:37733
I20251211 20:55:56.358834 9112 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.193:37733 every 8 connection(s)
I20251211 20:55:56.360810 8985 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
I20251211 20:55:56.368721 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 8985
I20251211 20:55:56.368916 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 8298
I20251211 20:55:56.388427 9113 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:56.388839 9113 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:56.389041 8423 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:55:56.389772 9113 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:56.392695 7230 ts_manager.cc:194] Re-registered known tserver with Master: 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:55:56.393762 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.193:60517
W20251211 20:55:56.543793 8342 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:55:56.544613 8343 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59318: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:55:56.545413 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 8298
I20251211 20:55:56.574409 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.196:35931
--local_ip_for_outbound_sockets=127.5.211.196
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=39639
--webserver_interface=127.5.211.196
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20251211 20:55:56.598613 8976 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:56.648550 9006 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Log is configured to *not* fsync() on all Append() calls
W20251211 20:55:57.017485 9121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:57.017951 9121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:57.018183 9121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:57.030136 9121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:57.030395 9121 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.196
I20251211 20:55:57.044425 9121 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.196:35931
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.5.211.196
--webserver_port=39639
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.196
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:57.046471 9121 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:57.048760 9121 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:57.060634 9127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:57.066314 9128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:57.068665 9130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:57.075490 9121 server_base.cc:1047] running on GCE node
I20251211 20:55:57.076332 9121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:57.077539 9121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:57.078826 9121 hybrid_clock.cc:648] HybridClock initialized: now 1765486557078764 us; error 53 us; skew 500 ppm
I20251211 20:55:57.079211 9121 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:57.082008 9121 webserver.cc:492] Webserver started at http://127.5.211.196:39639/ using document root <none> and password file <none>
I20251211 20:55:57.082928 9121 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:57.083045 9121 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:57.089627 9121 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20251211 20:55:57.092978 9136 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:57.094780 9121 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20251211 20:55:57.094915 9121 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
format_stamp: "Formatted at 2025-12-11 20:55:35 on dist-test-slave-sk6m"
I20251211 20:55:57.095767 9121 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:57.115664 9121 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:57.116499 9121 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:57.116796 9121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:57.117878 9121 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:57.120353 9143 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251211 20:55:57.128060 9121 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251211 20:55:57.128157 9121 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20251211 20:55:57.128320 9121 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251211 20:55:57.133263 9121 ts_tablet_manager.cc:616] Registered 1 tablets
I20251211 20:55:57.133409 9121 ts_tablet_manager.cc:595] Time spent register tablets: real 0.005s user 0.004s sys 0.000s
I20251211 20:55:57.133801 9143 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap starting.
W20251211 20:55:57.145149 7821 meta_cache.cc:302] tablet fb7cb7e3685a4401a6d872d961a7fb17: replica 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931) has failed: Network error: Client connection negotiation failed: client connection to 127.5.211.196:35931: connect: Connection refused (error 111)
I20251211 20:55:57.396669 9113 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:57.554378 7851 meta_cache.cc:1510] marking tablet server 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931) as failed
I20251211 20:55:57.556435 9143 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Log is configured to *not* fsync() on all Append() calls
I20251211 20:55:57.647608 7852 meta_cache.cc:1510] marking tablet server 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931) as failed
I20251211 20:55:57.882553 9250 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.196:35931 every 8 connection(s)
I20251211 20:55:57.882905 9121 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.196:35931
I20251211 20:55:57.884796 9121 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
I20251211 20:55:57.894551 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 9121
I20251211 20:55:57.894755 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 8440
I20251211 20:55:57.929931 8591 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:55:57.943992 9251 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:57.944474 9251 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:57.945789 9251 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:57.949692 7230 ts_manager.cc:194] Re-registered known tserver with Master: 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931)
I20251211 20:55:57.951609 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.196:43257
I20251211 20:55:58.106945 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 8440
I20251211 20:55:58.132230 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.195:37875
--local_ip_for_outbound_sockets=127.5.211.195
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=37277
--webserver_interface=127.5.211.195
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20251211 20:55:58.654652 9259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:55:58.658308 9259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:55:58.658541 9259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:55:58.673605 9259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:55:58.673895 9259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.195
I20251211 20:55:58.689401 9259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.195:37875
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.211.195
--webserver_port=37277
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.195
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:55:58.691670 9259 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:55:58.693840 9259 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:55:58.742389 9265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:58.742518 9264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:55:58.748760 9268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:55:58.749171 9259 server_base.cc:1047] running on GCE node
I20251211 20:55:58.750150 9259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:55:58.764070 9259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:55:58.765380 9259 hybrid_clock.cc:648] HybridClock initialized: now 1765486558765291 us; error 82 us; skew 500 ppm
I20251211 20:55:58.765828 9259 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:55:58.780932 9259 webserver.cc:492] Webserver started at http://127.5.211.195:37277/ using document root <none> and password file <none>
I20251211 20:55:58.781916 9259 fs_manager.cc:362] Metadata directory not provided
I20251211 20:55:58.782087 9259 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:55:58.795151 9259 fs_manager.cc:714] Time spent opening directory manager: real 0.011s user 0.002s sys 0.009s
I20251211 20:55:58.805423 9273 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:58.807060 9259 fs_manager.cc:730] Time spent opening block manager: real 0.010s user 0.005s sys 0.005s
I20251211 20:55:58.807310 9259 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
uuid: "671b0c18e41544439fd978aa70f7ff53"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:55:58.808283 9259 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:55:58.857179 9259 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:55:58.858340 9259 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:55:58.858879 9259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:55:58.866909 9259 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:55:58.875363 9259 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:55:58.875500 9259 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20251211 20:55:58.875643 9259 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:55:58.875733 9259 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:55:58.955878 9251 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:55:59.261094 8869 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap replayed 1/1 log segments. Stats: ops{read=901 overwritten=0 applied=900 ignored=0} inserts{seen=44900 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20251211 20:55:59.261907 8869 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap complete.
I20251211 20:55:59.263692 8869 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Time spent bootstrapping tablet: real 3.744s user 3.391s sys 0.268s
I20251211 20:55:59.270936 8869 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:59.281585 8869 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Initialized, Role: FOLLOWER
I20251211 20:55:59.282433 8869 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 900, Last appended: 3.901, Last appended by leader: 901, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:59.299430 8869 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Time spent starting tablet: real 0.036s user 0.009s sys 0.024s
I20251211 20:55:59.322502 9259 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.195:37875
I20251211 20:55:59.322562 9387 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.195:37875 every 8 connection(s)
I20251211 20:55:59.324364 9259 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
I20251211 20:55:59.328943 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 9259
I20251211 20:55:59.376919 9388 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:55:59.377338 9388 heartbeater.cc:461] Registering TS with master...
I20251211 20:55:59.378402 9388 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:55:59.380100 7230 ts_manager.cc:194] Re-registered known tserver with Master: 671b0c18e41544439fd978aa70f7ff53 (127.5.211.195:37875)
I20251211 20:55:59.381129 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.195:33271
W20251211 20:55:59.466470 8890 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:59.467404 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:59.482239 7851 scanner-internal.cc:458] Time spent opening tablet: real 5.843s user 0.007s sys 0.000s
I20251211 20:55:59.603978 9340 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:55:59.604218 9340 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:55:59.606022 9340 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:55:59.646782 9068 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 5 candidate_status { last_received { term: 3 index: 901 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" is_pre_election: true
W20251211 20:55:59.652204 8865 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Tablet error from VoteRequest() call to peer 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:55:59.656245 7852 scanner-internal.cc:458] Time spent opening tablet: real 6.030s user 0.003s sys 0.005s
I20251211 20:55:59.673327 9206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 5 candidate_status { last_received { term: 3 index: 901 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" is_pre_election: true
W20251211 20:55:59.676664 8864 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:55:59.676832 8864 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ccd26b0e8fe409f91fbd1da24bc7137; no voters: 30c15ab665fb453fbea8cda8fd8e13d0, 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:55:59.677515 9340 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20251211 20:55:59.808162 8886 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:55:59.809944 8886 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:56:00.028331 9340 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:56:00.028545 9340 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:00.029675 9068 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 5 candidate_status { last_received { term: 3 index: 901 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" is_pre_election: true
I20251211 20:56:00.029711 9206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 5 candidate_status { last_received { term: 3 index: 901 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" is_pre_election: true
I20251211 20:56:00.030299 9340 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
W20251211 20:56:00.030577 8865 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Tablet error from VoteRequest() call to peer 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:56:00.030855 8864 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:56:00.030953 8864 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ccd26b0e8fe409f91fbd1da24bc7137; no voters: 30c15ab665fb453fbea8cda8fd8e13d0, 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:56:00.031335 9340 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20251211 20:56:00.164134 8891 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:00.165346 8891 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:56:00.210887 9006 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=898 overwritten=0 applied=898 ignored=0} inserts{seen=44800 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:56:00.211645 9006 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap complete.
I20251211 20:56:00.213666 9006 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Time spent bootstrapping tablet: real 3.930s user 3.375s sys 0.281s
I20251211 20:56:00.218957 9006 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:00.219914 9006 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c15ab665fb453fbea8cda8fd8e13d0, State: Initialized, Role: FOLLOWER
I20251211 20:56:00.221244 9006 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 898, Last appended: 3.898, Last appended by leader: 898, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:00.239353 9006 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Time spent starting tablet: real 0.025s user 0.011s sys 0.012s
I20251211 20:56:00.357524 9340 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:56:00.357786 9340 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:00.358752 9340 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:56:00.360918 9206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 5 candidate_status { last_received { term: 3 index: 901 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" is_pre_election: true
W20251211 20:56:00.361541 8864 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:56:00.361642 9068 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 5 candidate_status { last_received { term: 3 index: 901 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" is_pre_election: true
I20251211 20:56:00.361930 9068 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 4.
I20251211 20:56:00.362810 8865 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0, 5ccd26b0e8fe409f91fbd1da24bc7137; no voters: 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:56:00.363071 9340 raft_consensus.cc:2804] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20251211 20:56:00.363214 9340 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:56:00.363350 9340 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 4 FOLLOWER]: Advancing to term 5
I20251211 20:56:00.369472 9340 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:00.370703 9206 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 5 candidate_status { last_received { term: 3 index: 901 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
W20251211 20:56:00.371308 8864 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:56:00.372216 9068 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 5 candidate_status { last_received { term: 3 index: 901 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
I20251211 20:56:00.372445 9068 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 4 FOLLOWER]: Advancing to term 5
I20251211 20:56:00.378027 9340 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 election: Requested vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:56:00.378626 9068 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 5.
I20251211 20:56:00.379150 8865 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0, 5ccd26b0e8fe409f91fbd1da24bc7137; no voters: 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:56:00.379778 9340 raft_consensus.cc:2804] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 5 FOLLOWER]: Leader election won for term 5
I20251211 20:56:00.384274 9388 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
W20251211 20:56:00.404207 9027 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:00.405315 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:56:00.418232 9340 raft_consensus.cc:697] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 5 LEADER]: Becoming Leader. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Running, Role: LEADER
I20251211 20:56:00.419096 9340 consensus_queue.cc:237] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 900, Committed index: 900, Last appended: 3.901, Last appended by leader: 901, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:00.483167 7226 catalog_manager.cc:5654] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } health_report { overall_health: UNKNOWN } } }
W20251211 20:56:00.499904 8864 consensus_peers.cc:597] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 -> Peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Couldn't send request to peer 6d21816e6f984a25a7eb64e5c8b29a2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:56:00.505872 9048 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:00.509685 9068 raft_consensus.cc:1275] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 5 FOLLOWER]: Refusing update from remote peer 5ccd26b0e8fe409f91fbd1da24bc7137: Log matching property violated. Preceding OpId in replica: term: 3 index: 898. Preceding OpId from leader: term: 5 index: 902. (index mismatch)
I20251211 20:56:00.510917 9340 consensus_queue.cc:1048] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 902, Last known committed idx: 898, Time since last communication: 0.000s
I20251211 20:56:00.523558 8911 tablet_service.cc:1467] Tablet server has 1 leaders and 2 scanners
I20251211 20:56:00.574218 9321 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:00.586323 9185 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:00.589011 9443 rpcz_store.cc:275] Call kudu.tserver.TabletServerService.Write from 127.0.0.1:54156 (ReqId={client: 441fb1938a8e4c3587371e21fa4e3a3e, seq_no=898, attempt_no=83}) took 1287 ms. Trace:
I20251211 20:56:00.589711 9443 rpcz_store.cc:276] 1211 20:55:59.301193 (+ 0us) service_pool.cc:168] Inserting onto call queue
1211 20:55:59.301356 (+ 163us) service_pool.cc:225] Handling call
1211 20:56:00.588961 (+1287605us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20251211 20:56:00.605273 9438 mvcc.cc:204] Tried to move back new op lower bound from 7231432951985111040 to 7231432951490170880. Current Snapshot: MvccSnapshot[applied={T|T < 7231432951985111040 or (T in {7231432951985111040})}]
W20251211 20:56:01.005398 8864 consensus_peers.cc:597] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 -> Peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Couldn't send request to peer 6d21816e6f984a25a7eb64e5c8b29a2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
W20251211 20:56:01.536590 8864 consensus_peers.cc:597] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 -> Peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Couldn't send request to peer 6d21816e6f984a25a7eb64e5c8b29a2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 11: this message will repeat every 5th retry.
I20251211 20:56:01.582938 9143 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=900 overwritten=0 applied=900 ignored=0} inserts{seen=44900 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:56:01.584241 9143 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap complete.
I20251211 20:56:01.586611 9143 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Time spent bootstrapping tablet: real 4.453s user 3.562s sys 0.247s
I20251211 20:56:01.591128 9143 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 4 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:01.593092 9143 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 4 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d21816e6f984a25a7eb64e5c8b29a2c, State: Initialized, Role: FOLLOWER
I20251211 20:56:01.594029 9143 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 900, Last appended: 3.900, Last appended by leader: 900, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:01.600148 9143 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Time spent starting tablet: real 0.013s user 0.001s sys 0.008s
W20251211 20:56:01.607188 7853 scanner-internal.cc:458] Time spent opening tablet: real 8.337s user 0.003s sys 0.004s
I20251211 20:56:01.625500 9206 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 4 FOLLOWER]: Advancing to term 5
I20251211 20:56:01.652473 9206 raft_consensus.cc:1275] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 5 FOLLOWER]: Refusing update from remote peer 5ccd26b0e8fe409f91fbd1da24bc7137: Log matching property violated. Preceding OpId in replica: term: 3 index: 900. Preceding OpId from leader: term: 3 index: 901. (index mismatch)
I20251211 20:56:01.656338 9420 consensus_queue.cc:1050] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 902, Last known committed idx: 900, Time since last communication: 0.000s
I20251211 20:56:02.173873 9454 mvcc.cc:204] Tried to move back new op lower bound from 7231432956350173184 to 7231432951490170880. Current Snapshot: MvccSnapshot[applied={T|T < 7231432956350173184}]
I20251211 20:56:07.285188 9048 tablet_service.cc:1467] Tablet server has 0 leaders and 3 scanners
I20251211 20:56:07.327594 8911 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251211 20:56:07.372814 9185 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:07.383059 9321 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:08.797438 7230 ts_manager.cc:284] Unset tserver state for 5ccd26b0e8fe409f91fbd1da24bc7137 from MAINTENANCE_MODE
I20251211 20:56:08.889528 8976 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:08.920272 9251 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:09.406600 9388 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:09.556031 7226 ts_manager.cc:284] Unset tserver state for 671b0c18e41544439fd978aa70f7ff53 from MAINTENANCE_MODE
I20251211 20:56:09.634431 9113 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:09.705431 7226 ts_manager.cc:284] Unset tserver state for 6d21816e6f984a25a7eb64e5c8b29a2c from MAINTENANCE_MODE
I20251211 20:56:09.894712 8976 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:09.925175 9251 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:09.957024 7226 ts_manager.cc:284] Unset tserver state for 30c15ab665fb453fbea8cda8fd8e13d0 from MAINTENANCE_MODE
I20251211 20:56:10.410892 9388 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:10.638922 9113 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:10.900331 8976 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:10.929116 9251 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:12.602465 7226 ts_manager.cc:295] Set tserver state for 671b0c18e41544439fd978aa70f7ff53 to MAINTENANCE_MODE
I20251211 20:56:13.626032 7226 ts_manager.cc:295] Set tserver state for 30c15ab665fb453fbea8cda8fd8e13d0 to MAINTENANCE_MODE
I20251211 20:56:13.643352 7226 ts_manager.cc:295] Set tserver state for 6d21816e6f984a25a7eb64e5c8b29a2c to MAINTENANCE_MODE
I20251211 20:56:13.750069 7226 ts_manager.cc:295] Set tserver state for 5ccd26b0e8fe409f91fbd1da24bc7137 to MAINTENANCE_MODE
I20251211 20:56:14.768134 9321 tablet_service.cc:1460] Tablet server 671b0c18e41544439fd978aa70f7ff53 set to quiescing
I20251211 20:56:14.768386 9321 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:15.753446 9048 tablet_service.cc:1460] Tablet server 30c15ab665fb453fbea8cda8fd8e13d0 set to quiescing
I20251211 20:56:15.753666 9048 tablet_service.cc:1467] Tablet server has 0 leaders and 3 scanners
I20251211 20:56:15.890369 8911 tablet_service.cc:1460] Tablet server 5ccd26b0e8fe409f91fbd1da24bc7137 set to quiescing
I20251211 20:56:15.890563 8911 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251211 20:56:15.901674 9420 raft_consensus.cc:993] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: : Instructing follower 6d21816e6f984a25a7eb64e5c8b29a2c to start an election
I20251211 20:56:15.901778 9420 raft_consensus.cc:1081] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 5 LEADER]: Signalling peer 6d21816e6f984a25a7eb64e5c8b29a2c to start an election
I20251211 20:56:15.902194 9655 raft_consensus.cc:993] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: : Instructing follower 6d21816e6f984a25a7eb64e5c8b29a2c to start an election
I20251211 20:56:15.902657 9655 raft_consensus.cc:1081] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 5 LEADER]: Signalling peer 6d21816e6f984a25a7eb64e5c8b29a2c to start an election
I20251211 20:56:15.903502 9205 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17"
dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
from {username='slave'} at 127.5.211.194:49571
I20251211 20:56:15.903726 9205 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20251211 20:56:15.903872 9205 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 5 FOLLOWER]: Advancing to term 6
I20251211 20:56:15.907457 9204 tablet_service.cc:2038] Received Run Leader Election RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17"
dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
from {username='slave'} at 127.5.211.194:49571
I20251211 20:56:15.908430 9205 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:15.911710 9204 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 6 FOLLOWER]: Starting forced leader election (received explicit request)
I20251211 20:56:15.911906 9204 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 6 FOLLOWER]: Advancing to term 7
I20251211 20:56:15.918706 9204 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:15.925616 9205 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 6 election: Requested vote from peers 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:56:15.920176 9204 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 7 election: Requested vote from peers 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:56:15.947966 9204 raft_consensus.cc:1240] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 7 FOLLOWER]: Rejecting Update request from peer 5ccd26b0e8fe409f91fbd1da24bc7137 for earlier term 5. Current term is 7. Ops: [5.2274-5.2274]
I20251211 20:56:15.948658 9420 consensus_queue.cc:1059] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 }, Status: INVALID_TERM, Last received: 5.2273, Next index: 2274, Last known committed idx: 2273, Time since last communication: 0.000s
I20251211 20:56:15.949015 9420 raft_consensus.cc:3055] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 5 LEADER]: Stepping down as leader of term 5
I20251211 20:56:15.949090 9420 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Running, Role: LEADER
I20251211 20:56:15.949369 9420 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2274, Committed index: 2274, Last appended: 5.2274, Last appended by leader: 2274, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:15.949921 9420 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 5 FOLLOWER]: Advancing to term 7
W20251211 20:56:15.956144 8887 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:15.956194 9438 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.010s ago.
W20251211 20:56:15.959268 9438 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: Cannot assign timestamp to op. Tablet is not in leader mode. Last heard from a leader: 0.013s ago.
W20251211 20:56:15.963729 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:15.963773 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:15.965577 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:15.971196 9025 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:15.972534 9025 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:15.972676 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:56:15.975543 9068 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" candidate_term: 7 candidate_status { last_received { term: 5 index: 2273 } } ignore_live_leader: true dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
I20251211 20:56:15.975732 9068 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 5 FOLLOWER]: Advancing to term 7
I20251211 20:56:15.978786 9068 raft_consensus.cc:2410] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6d21816e6f984a25a7eb64e5c8b29a2c for term 7 because replica has last-logged OpId of term: 5 index: 2274, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2273.
I20251211 20:56:15.979101 9067 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2273 } } ignore_live_leader: true dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
I20251211 20:56:15.979293 9067 raft_consensus.cc:2368] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6d21816e6f984a25a7eb64e5c8b29a2c for earlier term 6. Current term is 7.
I20251211 20:56:15.979908 9139 leader_election.cc:400] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 6 election: Vote denied by peer 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) with higher term. Message: Invalid argument: T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6d21816e6f984a25a7eb64e5c8b29a2c for earlier term 6. Current term is 7.
I20251211 20:56:15.980007 9139 leader_election.cc:403] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20251211 20:56:15.982216 8887 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:15.982523 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:15.982758 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:56:15.986238 8931 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" candidate_term: 6 candidate_status { last_received { term: 5 index: 2273 } } ignore_live_leader: true dest_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
I20251211 20:56:15.986601 8931 raft_consensus.cc:2368] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6d21816e6f984a25a7eb64e5c8b29a2c for earlier term 6. Current term is 7.
I20251211 20:56:15.987246 9138 leader_election.cc:400] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 6 election: Vote denied by peer 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381) with higher term. Message: Invalid argument: T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6d21816e6f984a25a7eb64e5c8b29a2c for earlier term 6. Current term is 7.
I20251211 20:56:15.987563 8931 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" candidate_term: 7 candidate_status { last_received { term: 5 index: 2273 } } ignore_live_leader: true dest_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
I20251211 20:56:15.987831 8931 raft_consensus.cc:2410] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6d21816e6f984a25a7eb64e5c8b29a2c for term 7 because replica has last-logged OpId of term: 5 index: 2274, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 2273.
I20251211 20:56:15.988641 9138 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d21816e6f984a25a7eb64e5c8b29a2c; no voters: 30c15ab665fb453fbea8cda8fd8e13d0, 5ccd26b0e8fe409f91fbd1da24bc7137
W20251211 20:56:15.989578 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:15.990310 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:16.021270 9692 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) with higher term. Message: Invalid argument: T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6d21816e6f984a25a7eb64e5c8b29a2c for earlier term 6. Current term is 7.
I20251211 20:56:16.021661 9692 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20251211 20:56:16.023001 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.029091 9139 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.5.211.193:37733, user_credentials={real_user=slave}} blocked reactor thread for 49293.2us
W20251211 20:56:16.030725 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.032802 9025 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.033632 9025 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.042804 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.044101 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.046649 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.056316 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:16.057044 9185 tablet_service.cc:1460] Tablet server 6d21816e6f984a25a7eb64e5c8b29a2c set to quiescing
I20251211 20:56:16.057242 9185 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251211 20:56:16.058494 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.059154 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.070158 9025 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.070891 9024 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.075162 9025 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.084486 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.087221 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.091913 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.100586 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.102557 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.108331 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.118721 9025 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.121568 9024 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.127575 9024 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.139500 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.140650 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.146592 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.156121 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.158514 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.163048 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.174692 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.175096 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.179019 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.194319 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.197340 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.198587 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.215447 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.217916 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.219367 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.229800 9655 raft_consensus.cc:670] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: failed to trigger leader election: Illegal state: leader elections are disabled
W20251211 20:56:16.237157 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.237425 9024 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.238480 9024 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.244449 9694 raft_consensus.cc:670] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: failed to trigger leader election: Illegal state: leader elections are disabled
W20251211 20:56:16.262473 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.263653 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.264240 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.286230 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.286175 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.288924 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.309441 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.312911 9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.315448 9024 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.335740 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.340569 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.342463 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.362123 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.366659 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.367975 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.391269 9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.392582 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.393592 9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.422343 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.423404 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.424736 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.450466 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.454202 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.454298 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.480875 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.487164 9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.492180 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.515065 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.521035 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.522423 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.548257 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.553735 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.554903 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.582242 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.587116 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.590631 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.610073 9695 raft_consensus.cc:670] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: failed to trigger leader election: Illegal state: leader elections are disabled
W20251211 20:56:16.618173 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.623052 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.625932 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.652444 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.656883 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.659384 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.690934 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.691057 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.697170 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.730875 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.731608 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.734833 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.769496 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.770840 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.774194 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.810793 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.810793 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.813084 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.853085 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.854817 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.855388 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.894073 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.896868 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.897090 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:16.936698 9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.937911 9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.938979 9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.979697 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.982667 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:16.985356 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.026096 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.026314 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.027418 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.071828 9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.072062 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.073380 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.146015 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.147575 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.149235 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.195706 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.196908 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.197577 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.243259 9024 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.243280 9023 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.243283 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.292811 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.294222 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.295845 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.344699 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.346537 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.346663 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.397799 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.399677 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.400887 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.452185 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.455925 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.456514 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.506523 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.511777 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.512835 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:17.529644 8911 tablet_service.cc:1460] Tablet server 5ccd26b0e8fe409f91fbd1da24bc7137 set to quiescing
I20251211 20:56:17.529704 9048 tablet_service.cc:1460] Tablet server 30c15ab665fb453fbea8cda8fd8e13d0 set to quiescing
I20251211 20:56:17.529848 8911 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:17.529902 9048 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
W20251211 20:56:17.558759 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.564752 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.565922 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.613476 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.621356 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.622517 8889 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.667357 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:17.675212 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 8847
W20251211 20:56:17.678351 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:17.678503 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:17.682133 8970 generic_service.cc:196] Checking for leaks (request via RPC)
W20251211 20:56:17.725466 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.736972 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.737087 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:56:17.917538 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 8847
W20251211 20:56:17.917614 8888 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:54156: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:17.962785 7821 meta_cache.cc:302] tablet fb7cb7e3685a4401a6d872d961a7fb17: replica 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381) has failed: Network error: recv got EOF from 127.5.211.194:45381 (error 108)
I20251211 20:56:17.963446 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.194:45381
--local_ip_for_outbound_sockets=127.5.211.194
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=45957
--webserver_interface=127.5.211.194
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20251211 20:56:17.992156 7852 meta_cache.cc:1510] marking tablet server 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381) as failed
W20251211 20:56:18.021390 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:18.022667 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:18.081970 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:18.082026 9028 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:18.082037 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:18.141332 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:18.207129 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:18.265543 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:18.267239 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:18.271417 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:18.320595 9722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:56:18.320861 9722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:56:18.320945 9722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:56:18.329010 9722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:56:18.329157 9722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.194
W20251211 20:56:18.330860 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:18.333959 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:56:18.339865 9722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.194:45381
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.211.194
--webserver_port=45957
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.194
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:56:18.341323 9722 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:56:18.342994 9722 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:56:18.352252 9731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:56:18.352236 9728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:56:18.352258 9729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:56:18.353222 9722 server_base.cc:1047] running on GCE node
I20251211 20:56:18.353753 9722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:56:18.354619 9722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:56:18.355829 9722 hybrid_clock.cc:648] HybridClock initialized: now 1765486578355785 us; error 41 us; skew 500 ppm
I20251211 20:56:18.356097 9722 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:56:18.358161 9722 webserver.cc:492] Webserver started at http://127.5.211.194:45957/ using document root <none> and password file <none>
I20251211 20:56:18.358850 9722 fs_manager.cc:362] Metadata directory not provided
I20251211 20:56:18.358950 9722 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:56:18.363976 9722 fs_manager.cc:714] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20251211 20:56:18.367095 9737 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:56:18.368499 9722 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20251211 20:56:18.368670 9722 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
uuid: "5ccd26b0e8fe409f91fbd1da24bc7137"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:56:18.369576 9722 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:56:18.399012 9722 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:56:18.399884 9722 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:56:18.400202 9722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:56:18.401199 9722 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:56:18.403522 9744 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
W20251211 20:56:18.404488 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:18.409593 9722 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251211 20:56:18.409675 9722 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.007s user 0.001s sys 0.000s
I20251211 20:56:18.409792 9722 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251211 20:56:18.412675 9722 ts_tablet_manager.cc:616] Registered 1 tablets
I20251211 20:56:18.412786 9722 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20251211 20:56:18.413288 9744 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap starting.
I20251211 20:56:18.451260 9722 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.194:45381
I20251211 20:56:18.451325 9850 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.194:45381 every 8 connection(s)
I20251211 20:56:18.452828 9722 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-1/data/info.pb
I20251211 20:56:18.456534 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 9722
I20251211 20:56:18.456758 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 8985
W20251211 20:56:18.461694 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:18.466730 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:18.470676 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:56:18.474449 9851 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:56:18.474828 9851 heartbeater.cc:461] Registering TS with master...
I20251211 20:56:18.475805 9851 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:18.476523 9108 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:56:18.479521 7230 ts_manager.cc:194] Re-registered known tserver with Master: 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381)
I20251211 20:56:18.480569 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.194:42503
W20251211 20:56:18.623564 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:18.722885 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 8985
W20251211 20:56:18.723030 9026 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:39834: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:18.761044 7217 connection.cc:537] server connection from 127.5.211.193:60517 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:56:18.761499 7851 meta_cache.cc:1510] marking tablet server 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) as failed
I20251211 20:56:18.761793 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.193:37733
--local_ip_for_outbound_sockets=127.5.211.193
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=41527
--webserver_interface=127.5.211.193
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
W20251211 20:56:18.970584 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:18.972465 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:19.043207 7822 meta_cache.cc:302] tablet fb7cb7e3685a4401a6d872d961a7fb17: replica 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) has failed: Network error: Client connection negotiation failed: client connection to 127.5.211.193:37733: connect: Connection refused (error 111)
W20251211 20:56:19.046216 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:19.067004 9744 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Log is configured to *not* fsync() on all Append() calls
W20251211 20:56:19.118692 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:19.121948 7853 meta_cache.cc:1510] marking tablet server 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) as failed
W20251211 20:56:19.123184 9859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:56:19.123525 9859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:56:19.123663 9859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:56:19.132933 9859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:56:19.133131 9859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.193
I20251211 20:56:19.142547 9859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.193:37733
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.211.193
--webserver_port=41527
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.193
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:56:19.144032 9859 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:56:19.145819 9859 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:56:19.154584 9865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:56:19.156278 9866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:56:19.157862 9859 server_base.cc:1047] running on GCE node
W20251211 20:56:19.158105 9868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:56:19.158833 9859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:56:19.159885 9859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:56:19.161113 9859 hybrid_clock.cc:648] HybridClock initialized: now 1765486579161066 us; error 38 us; skew 500 ppm
I20251211 20:56:19.161441 9859 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:56:19.163997 9859 webserver.cc:492] Webserver started at http://127.5.211.193:41527/ using document root <none> and password file <none>
I20251211 20:56:19.165059 9859 fs_manager.cc:362] Metadata directory not provided
I20251211 20:56:19.165241 9859 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:56:19.171655 9859 fs_manager.cc:714] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20251211 20:56:19.174786 9874 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:56:19.176065 9859 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20251211 20:56:19.176201 9859 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:56:19.176805 9859 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:56:19.194813 9859 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:56:19.195622 9859 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:56:19.195997 9859 kserver.cc:163] Server-wide thread pool size limit: 3276
W20251211 20:56:19.196795 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:19.197801 9859 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:56:19.198429 7852 meta_cache.cc:1510] marking tablet server 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) as failed
I20251211 20:56:19.200587 9881 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251211 20:56:19.208446 9859 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251211 20:56:19.208539 9859 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.008s user 0.001s sys 0.000s
I20251211 20:56:19.208662 9859 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251211 20:56:19.211381 9859 ts_tablet_manager.cc:616] Registered 1 tablets
I20251211 20:56:19.211475 9859 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20251211 20:56:19.211962 9881 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap starting.
W20251211 20:56:19.263471 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:19.287343 9859 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.193:37733
I20251211 20:56:19.287469 9987 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.193:37733 every 8 connection(s)
I20251211 20:56:19.289475 9859 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-0/data/info.pb
I20251211 20:56:19.293869 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 9859
I20251211 20:56:19.294088 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 9121
W20251211 20:56:19.341439 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:19.357923 9245 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:56:19.363234 9988 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:56:19.363593 9988 heartbeater.cc:461] Registering TS with master...
I20251211 20:56:19.364557 9988 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:19.367604 7230 ts_manager.cc:194] Re-registered known tserver with Master: 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:56:19.369158 7230 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.193:37769
I20251211 20:56:19.482826 9851 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
W20251211 20:56:19.722288 9165 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:19.731454 9164 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
W20251211 20:56:19.732189 9163 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:47420: Illegal state: replica 6d21816e6f984a25a7eb64e5c8b29a2c is not leader of this config: current role FOLLOWER
I20251211 20:56:19.733784 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 9121
I20251211 20:56:19.792390 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.196:35931
--local_ip_for_outbound_sockets=127.5.211.196
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=39639
--webserver_interface=127.5.211.196
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20251211 20:56:19.983580 7851 meta_cache.cc:1510] marking tablet server 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931) as failed
W20251211 20:56:20.073019 7821 meta_cache.cc:302] tablet fb7cb7e3685a4401a6d872d961a7fb17: replica 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931) has failed: Network error: Client connection negotiation failed: client connection to 127.5.211.196:35931: connect: Connection refused (error 111)
I20251211 20:56:20.211689 9881 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Log is configured to *not* fsync() on all Append() calls
W20251211 20:56:20.257947 9996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:56:20.258325 9996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:56:20.258491 9996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:56:20.271026 9996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:56:20.271225 9996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.196
I20251211 20:56:20.285861 9996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.196:35931
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
--webserver_interface=127.5.211.196
--webserver_port=39639
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.196
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:56:20.287791 9996 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:56:20.290076 9996 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:56:20.304431 10003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:56:20.310318 10004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:56:20.312958 10006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:56:20.313294 9996 server_base.cc:1047] running on GCE node
I20251211 20:56:20.314146 9996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:56:20.315150 9996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:56:20.316437 9996 hybrid_clock.cc:648] HybridClock initialized: now 1765486580316385 us; error 47 us; skew 500 ppm
I20251211 20:56:20.316737 9996 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:56:20.321065 9996 webserver.cc:492] Webserver started at http://127.5.211.196:39639/ using document root <none> and password file <none>
I20251211 20:56:20.322068 9996 fs_manager.cc:362] Metadata directory not provided
I20251211 20:56:20.322181 9996 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:56:20.334753 9996 fs_manager.cc:714] Time spent opening directory manager: real 0.010s user 0.007s sys 0.004s
I20251211 20:56:20.337401 10012 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:56:20.338693 9996 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20251211 20:56:20.338852 9996 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
format_stamp: "Formatted at 2025-12-11 20:55:35 on dist-test-slave-sk6m"
I20251211 20:56:20.339507 9996 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:56:20.352038 9996 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:56:20.352726 9996 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:56:20.352957 9996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:56:20.353792 9996 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:56:20.355937 10019 ts_tablet_manager.cc:548] Loading tablet metadata (0/1 complete)
I20251211 20:56:20.361209 9996 ts_tablet_manager.cc:585] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20251211 20:56:20.361299 9996 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20251211 20:56:20.361419 9996 ts_tablet_manager.cc:600] Registering tablets (0/1 complete)
I20251211 20:56:20.364048 9996 ts_tablet_manager.cc:616] Registered 1 tablets
I20251211 20:56:20.364187 9996 ts_tablet_manager.cc:595] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20251211 20:56:20.364688 10019 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap starting.
I20251211 20:56:20.373049 9988 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:56:20.730177 7853 meta_cache.cc:1510] marking tablet server 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931) as failed
I20251211 20:56:20.807354 7852 meta_cache.cc:1510] marking tablet server 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931) as failed
I20251211 20:56:21.081003 10125 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.196:35931 every 8 connection(s)
I20251211 20:56:21.081338 9996 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.196:35931
I20251211 20:56:21.083395 9996 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-3/data/info.pb
I20251211 20:56:21.092487 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 9996
I20251211 20:56:21.092707 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 9259
I20251211 20:56:21.124218 10126 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:56:21.124603 10126 heartbeater.cc:461] Registering TS with master...
I20251211 20:56:21.125633 10126 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:21.128527 7225 ts_manager.cc:194] Re-registered known tserver with Master: 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931)
I20251211 20:56:21.130142 7225 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.196:57025
I20251211 20:56:21.130221 9383 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:56:21.320482 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 9259
I20251211 20:56:21.347922 5967 external_mini_cluster.cc:1366] Running /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
/tmp/dist-test-taskrKV6hb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.211.195:37875
--local_ip_for_outbound_sockets=127.5.211.195
--tserver_master_addrs=127.5.211.254:43355
--webserver_port=37277
--webserver_interface=127.5.211.195
--builtin_ntp_servers=127.5.211.212:37141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--scanner_default_batch_size_bytes=100 with env {}
I20251211 20:56:21.642741 10019 log.cc:826] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Log is configured to *not* fsync() on all Append() calls
W20251211 20:56:21.847780 10134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20251211 20:56:21.848269 10134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20251211 20:56:21.848469 10134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20251211 20:56:21.861277 10134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20251211 20:56:21.861564 10134 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.211.195
I20251211 20:56:21.876420 10134 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.211.212:37141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--raft_heartbeat_interval_ms=100
--fs_data_dirs=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.211.195:37875
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.211.195
--webserver_port=37277
--tserver_master_addrs=127.5.211.254:43355
--scanner_default_batch_size_bytes=100
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.211.195
--log_dir=/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.19.0-SNAPSHOT
revision 468897755969d686fb08386f42f7835685e7953a
build type FASTDEBUG
built by None at 11 Dec 2025 20:43:24 UTC on 24a791456cd2
build id 9468
ASAN enabled
I20251211 20:56:21.878690 10134 env_posix.cc:2267] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20251211 20:56:21.880995 10134 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20251211 20:56:21.904270 10140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:56:21.918305 10141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20251211 20:56:21.922151 10144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20251211 20:56:21.926348 10134 server_base.cc:1047] running on GCE node
I20251211 20:56:21.927219 10134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20251211 20:56:21.933830 10134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20251211 20:56:21.935161 10134 hybrid_clock.cc:648] HybridClock initialized: now 1765486581935080 us; error 75 us; skew 500 ppm
I20251211 20:56:21.935578 10134 server_base.cc:847] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20251211 20:56:21.953032 10134 webserver.cc:492] Webserver started at http://127.5.211.195:37277/ using document root <none> and password file <none>
I20251211 20:56:21.954197 10134 fs_manager.cc:362] Metadata directory not provided
I20251211 20:56:21.954406 10134 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20251211 20:56:21.968030 10134 fs_manager.cc:714] Time spent opening directory manager: real 0.011s user 0.007s sys 0.004s
I20251211 20:56:21.982161 10149 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20251211 20:56:21.983942 10134 fs_manager.cc:730] Time spent opening block manager: real 0.014s user 0.010s sys 0.001s
I20251211 20:56:21.984179 10134 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data,/tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
uuid: "671b0c18e41544439fd978aa70f7ff53"
format_stamp: "Formatted at 2025-12-11 20:55:34 on dist-test-slave-sk6m"
I20251211 20:56:21.985209 10134 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20251211 20:56:22.026785 10134 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20251211 20:56:22.027896 10134 env_posix.cc:2267] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20251211 20:56:22.028239 10134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20251211 20:56:22.044697 10134 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20251211 20:56:22.058880 10134 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20251211 20:56:22.059046 10134 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20251211 20:56:22.059204 10134 ts_tablet_manager.cc:616] Registered 0 tablets
I20251211 20:56:22.059294 10134 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20251211 20:56:22.133072 10126 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:56:22.828972 10261 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.5.211.195:37875 every 8 connection(s)
I20251211 20:56:22.829286 10134 rpc_server.cc:307] RPC server started. Bound to: 127.5.211.195:37875
I20251211 20:56:22.831275 10134 server_base.cc:1179] Dumped server information to /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0/minicluster-data/ts-2/data/info.pb
I20251211 20:56:22.839457 5967 external_mini_cluster.cc:1428] Started /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu as pid 10134
I20251211 20:56:22.881052 10262 heartbeater.cc:344] Connected to a master server at 127.5.211.254:43355
I20251211 20:56:22.881461 10262 heartbeater.cc:461] Registering TS with master...
I20251211 20:56:22.882480 10262 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:22.884276 7225 ts_manager.cc:194] Re-registered known tserver with Master: 671b0c18e41544439fd978aa70f7ff53 (127.5.211.195:37875)
I20251211 20:56:22.885275 7225 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.5.211.195:60791
I20251211 20:56:23.706326 9923 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:23.727129 10197 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:23.732049 10061 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:23.755733 9786 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:23.888439 10262 heartbeater.cc:499] Master 127.5.211.254:43355 was elected leader, sending a full tablet report...
I20251211 20:56:24.071161 9744 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap replaying log segment 1/1 (2.30M/3.93M this segment, stats: ops{read=1331 overwritten=0 applied=1330 ignored=0} inserts{seen=66350 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20251211 20:56:25.216017 9881 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap replaying log segment 1/1 (2.25M/3.90M this segment, stats: ops{read=1310 overwritten=0 applied=1310 ignored=0} inserts{seen=65350 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20251211 20:56:26.651824 10019 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap replaying log segment 1/1 (2.25M/3.87M this segment, stats: ops{read=1326 overwritten=0 applied=1325 ignored=0} inserts{seen=66100 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0)
I20251211 20:56:27.340660 9744 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap replayed 1/1 log segments. Stats: ops{read=2274 overwritten=0 applied=2274 ignored=0} inserts{seen=113550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:56:27.341440 9744 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Bootstrap complete.
I20251211 20:56:27.343761 9744 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Time spent bootstrapping tablet: real 8.931s user 8.121s sys 0.493s
I20251211 20:56:27.350891 9744 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:27.351744 9744 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Initialized, Role: FOLLOWER
I20251211 20:56:27.352506 9744 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2274, Last appended: 5.2274, Last appended by leader: 2274, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:27.354094 9744 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137: Time spent starting tablet: real 0.010s user 0.012s sys 0.000s
W20251211 20:56:27.413712 9766 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48640: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
W20251211 20:56:27.621619 9766 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48640: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:56:27.681505 10301 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:56:27.681809 10301 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:27.689859 10301 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:56:27.706960 9943 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 8 candidate_status { last_received { term: 5 index: 2274 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" is_pre_election: true
W20251211 20:56:27.710222 9740 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 pre-election: Tablet error from VoteRequest() call to peer 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733): Illegal state: must be running to vote when last-logged opid is not known
W20251211 20:56:27.710376 9766 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48640: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:56:27.713150 10081 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 8 candidate_status { last_received { term: 5 index: 2274 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" is_pre_election: true
W20251211 20:56:27.716372 9739 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 pre-election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:56:27.716524 9739 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5ccd26b0e8fe409f91fbd1da24bc7137; no voters: 30c15ab665fb453fbea8cda8fd8e13d0, 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:56:27.716953 10301 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20251211 20:56:27.858982 9766 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48640: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:56:27.942425 7853 scanner-internal.cc:172] Scan of tablet fb7cb7e3685a4401a6d872d961a7fb17 at 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) deadline exceeded after 10.000s
W20251211 20:56:27.942771 7853 scanner-internal.cc:458] Time spent opening tablet: real 11.300s user 0.010s sys 0.001s
I20251211 20:56:27.973191 9881 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2274 overwritten=0 applied=2274 ignored=0} inserts{seen=113550 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:56:27.973716 9881 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Bootstrap complete.
I20251211 20:56:27.975430 9881 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Time spent bootstrapping tablet: real 8.764s user 7.933s sys 0.519s
I20251211 20:56:27.978894 9881 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:27.979453 9881 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30c15ab665fb453fbea8cda8fd8e13d0, State: Initialized, Role: FOLLOWER
I20251211 20:56:27.980273 9881 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2274, Last appended: 5.2274, Last appended by leader: 2274, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:27.981765 9881 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20251211 20:56:28.010246 9902 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59918: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:28.010179 9903 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59918: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
I20251211 20:56:28.016737 7852 scanner-internal.cc:172] Scan of tablet fb7cb7e3685a4401a6d872d961a7fb17 at 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) deadline exceeded after 10.000s
W20251211 20:56:28.016995 7852 scanner-internal.cc:458] Time spent opening tablet: real 11.238s user 0.008s sys 0.005s
W20251211 20:56:28.073818 9766 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48640: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:56:28.161151 10301 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20251211 20:56:28.161327 10301 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:28.162006 10301 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:56:28.162426 10081 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 8 candidate_status { last_received { term: 5 index: 2274 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" is_pre_election: true
I20251211 20:56:28.162591 9943 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 8 candidate_status { last_received { term: 5 index: 2274 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" is_pre_election: true
I20251211 20:56:28.162914 9943 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 7.
W20251211 20:56:28.163085 9766 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:48640: Illegal state: replica 5ccd26b0e8fe409f91fbd1da24bc7137 is not leader of this config: current role FOLLOWER
I20251211 20:56:28.163342 9740 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0, 5ccd26b0e8fe409f91fbd1da24bc7137; no voters:
I20251211 20:56:28.163699 10301 raft_consensus.cc:2804] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20251211 20:56:28.163738 9739 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 pre-election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:56:28.163805 10301 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20251211 20:56:28.163890 10301 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 7 FOLLOWER]: Advancing to term 8
I20251211 20:56:28.168332 10301 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:28.169267 10081 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 8 candidate_status { last_received { term: 5 index: 2274 } } ignore_live_leader: false dest_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c"
I20251211 20:56:28.169355 9943 tablet_service.cc:1911] Received RequestConsensusVote() RPC: tablet_id: "fb7cb7e3685a4401a6d872d961a7fb17" candidate_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" candidate_term: 8 candidate_status { last_received { term: 5 index: 2274 } } ignore_live_leader: false dest_uuid: "30c15ab665fb453fbea8cda8fd8e13d0"
I20251211 20:56:28.169538 9943 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 7 FOLLOWER]: Advancing to term 8
W20251211 20:56:28.169766 9739 leader_election.cc:343] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 election: Tablet error from VoteRequest() call to peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Illegal state: must be running to vote when last-logged opid is not known
I20251211 20:56:28.169958 10301 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 election: Requested vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
I20251211 20:56:28.174059 9943 raft_consensus.cc:2468] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ccd26b0e8fe409f91fbd1da24bc7137 in term 8.
I20251211 20:56:28.174417 9740 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 30c15ab665fb453fbea8cda8fd8e13d0, 5ccd26b0e8fe409f91fbd1da24bc7137; no voters: 6d21816e6f984a25a7eb64e5c8b29a2c
I20251211 20:56:28.174825 10301 raft_consensus.cc:2804] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 8 FOLLOWER]: Leader election won for term 8
I20251211 20:56:28.175239 10301 raft_consensus.cc:697] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [term 8 LEADER]: Becoming Leader. State: Replica: 5ccd26b0e8fe409f91fbd1da24bc7137, State: Running, Role: LEADER
I20251211 20:56:28.176090 10301 consensus_queue.cc:237] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2274, Committed index: 2274, Last appended: 5.2274, Last appended by leader: 2274, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:28.180641 7225 catalog_manager.cc:5654] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } health_report { overall_health: UNKNOWN } } }
W20251211 20:56:28.282331 9739 consensus_peers.cc:597] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 -> Peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Couldn't send request to peer 6d21816e6f984a25a7eb64e5c8b29a2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20251211 20:56:28.301255 9943 raft_consensus.cc:1275] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 8 FOLLOWER]: Refusing update from remote peer 5ccd26b0e8fe409f91fbd1da24bc7137: Log matching property violated. Preceding OpId in replica: term: 5 index: 2274. Preceding OpId from leader: term: 8 index: 2275. (index mismatch)
I20251211 20:56:28.302208 10301 consensus_queue.cc:1048] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2275, Last known committed idx: 2274, Time since last communication: 0.000s
W20251211 20:56:28.376621 9903 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59918: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:28.381948 9903 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59918: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:28.465849 9903 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59918: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:28.473037 9903 tablet_service.cc:731] failed op from {username='slave'} at 127.0.0.1:59918: Illegal state: replica 30c15ab665fb453fbea8cda8fd8e13d0 is not leader of this config: current role FOLLOWER
W20251211 20:56:28.773984 9739 consensus_peers.cc:597] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 -> Peer 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931): Couldn't send request to peer 6d21816e6f984a25a7eb64e5c8b29a2c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20251211 20:56:28.803599 7851 scanner-internal.cc:172] Scan of tablet fb7cb7e3685a4401a6d872d961a7fb17 at 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733) deadline exceeded after 10.000s
W20251211 20:56:28.803807 7851 scanner-internal.cc:458] Time spent opening tablet: real 11.310s user 0.008s sys 0.004s
I20251211 20:56:29.239228 10019 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2273 overwritten=0 applied=2273 ignored=0} inserts{seen=113500 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20251211 20:56:29.240048 10019 tablet_bootstrap.cc:492] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Bootstrap complete.
I20251211 20:56:29.242106 10019 ts_tablet_manager.cc:1403] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Time spent bootstrapping tablet: real 8.878s user 8.015s sys 0.485s
I20251211 20:56:29.247023 10019 raft_consensus.cc:359] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 7 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:29.247987 10019 raft_consensus.cc:740] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 7 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d21816e6f984a25a7eb64e5c8b29a2c, State: Initialized, Role: FOLLOWER
I20251211 20:56:29.248692 10019 consensus_queue.cc:260] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2273, Last appended: 5.2273, Last appended by leader: 2273, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:29.253157 10019 ts_tablet_manager.cc:1434] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20251211 20:56:29.275785 10081 raft_consensus.cc:3060] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 7 FOLLOWER]: Advancing to term 8
I20251211 20:56:29.284073 10081 raft_consensus.cc:1275] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 8 FOLLOWER]: Refusing update from remote peer 5ccd26b0e8fe409f91fbd1da24bc7137: Log matching property violated. Preceding OpId in replica: term: 5 index: 2273. Preceding OpId from leader: term: 5 index: 2274. (index mismatch)
I20251211 20:56:29.286919 10301 consensus_queue.cc:1050] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 [LEADER]: Got LMP mismatch error from peer: Peer: permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2275, Last known committed idx: 2273, Time since last communication: 0.000s
I20251211 20:56:29.494206 10328 mvcc.cc:204] Tried to move back new op lower bound from 7231433066145800192 to 7231433065173569536. Current Snapshot: MvccSnapshot[applied={T|T < 7231433066145800192 or (T in {7231433066145800192})}]
I20251211 20:56:30.118736 9923 tablet_service.cc:1467] Tablet server has 0 leaders and 2 scanners
I20251211 20:56:30.210925 9786 tablet_service.cc:1467] Tablet server has 1 leaders and 0 scanners
I20251211 20:56:30.210994 10197 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:30.246114 10061 tablet_service.cc:1467] Tablet server has 0 leaders and 0 scanners
I20251211 20:56:31.861006 7225 ts_manager.cc:284] Unset tserver state for 30c15ab665fb453fbea8cda8fd8e13d0 from MAINTENANCE_MODE
I20251211 20:56:31.880697 7230 ts_manager.cc:284] Unset tserver state for 6d21816e6f984a25a7eb64e5c8b29a2c from MAINTENANCE_MODE
I20251211 20:56:31.887246 7226 ts_manager.cc:284] Unset tserver state for 5ccd26b0e8fe409f91fbd1da24bc7137 from MAINTENANCE_MODE
I20251211 20:56:31.904742 7226 ts_manager.cc:284] Unset tserver state for 671b0c18e41544439fd978aa70f7ff53 from MAINTENANCE_MODE
I20251211 20:56:31.908846 10262 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:32.334609 9988 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:32.535028 9851 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
I20251211 20:56:32.538347 10126 heartbeater.cc:507] Master 127.5.211.254:43355 requested a full tablet report, sending...
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/maintenance_mode-itest.cc:750: Failure
Value of: s.ok()
Actual: true
Expected: false
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/util/test_util.cc:403: Failure
Failed
Timed out waiting for assertion to pass.
I20251211 20:56:33.334889 5967 external_mini_cluster-itest-base.cc:80] Found fatal failure
I20251211 20:56:33.335377 5967 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 0 with UUID 30c15ab665fb453fbea8cda8fd8e13d0 and pid 9859
************************ BEGIN STACKS **************************
[New LWP 9861]
[New LWP 9862]
[New LWP 9863]
[New LWP 9864]
[New LWP 9870]
[New LWP 9871]
[New LWP 9872]
[New LWP 9875]
[New LWP 9876]
[New LWP 9877]
[New LWP 9878]
[New LWP 9879]
[New LWP 9880]
[New LWP 9882]
[New LWP 9883]
[New LWP 9884]
[New LWP 9885]
[New LWP 9886]
[New LWP 9887]
[New LWP 9888]
[New LWP 9889]
[New LWP 9890]
[New LWP 9891]
[New LWP 9892]
[New LWP 9893]
[New LWP 9894]
[New LWP 9895]
[New LWP 9896]
[New LWP 9897]
[New LWP 9898]
[New LWP 9899]
[New LWP 9900]
[New LWP 9901]
[New LWP 9902]
[New LWP 9903]
[New LWP 9904]
[New LWP 9905]
[New LWP 9906]
[New LWP 9907]
[New LWP 9908]
[New LWP 9909]
[New LWP 9910]
[New LWP 9911]
[New LWP 9912]
[New LWP 9913]
[New LWP 9914]
[New LWP 9915]
[New LWP 9916]
[New LWP 9917]
[New LWP 9918]
[New LWP 9919]
[New LWP 9920]
[New LWP 9921]
[New LWP 9922]
[New LWP 9923]
[New LWP 9924]
[New LWP 9925]
[New LWP 9926]
[New LWP 9927]
[New LWP 9928]
[New LWP 9929]
[New LWP 9930]
[New LWP 9931]
[New LWP 9932]
[New LWP 9933]
[New LWP 9934]
[New LWP 9935]
[New LWP 9936]
[New LWP 9937]
[New LWP 9938]
[New LWP 9939]
[New LWP 9940]
[New LWP 9941]
[New LWP 9942]
[New LWP 9943]
[New LWP 9944]
[New LWP 9945]
[New LWP 9946]
[New LWP 9947]
[New LWP 9948]
[New LWP 9949]
[New LWP 9950]
[New LWP 9951]
[New LWP 9952]
[New LWP 9953]
[New LWP 9954]
[New LWP 9955]
[New LWP 9956]
[New LWP 9957]
[New LWP 9958]
[New LWP 9959]
[New LWP 9960]
[New LWP 9961]
[New LWP 9962]
[New LWP 9963]
[New LWP 9964]
[New LWP 9965]
[New LWP 9966]
[New LWP 9967]
[New LWP 9968]
[New LWP 9969]
[New LWP 9970]
[New LWP 9971]
[New LWP 9972]
[New LWP 9973]
[New LWP 9974]
[New LWP 9975]
[New LWP 9976]
[New LWP 9977]
[New LWP 9978]
[New LWP 9979]
[New LWP 9980]
[New LWP 9981]
[New LWP 9982]
[New LWP 9983]
[New LWP 9984]
[New LWP 9985]
[New LWP 9986]
[New LWP 9987]
[New LWP 9988]
[New LWP 9989]
[New LWP 10314]
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c72
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c72
0x00007fd0eb1b2d50 in ?? ()
Id Target Id Frame
* 1 LWP 9859 "kudu" 0x00007fd0eb1b2d50 in ?? ()
2 LWP 9861 "kudu" 0x00007fd0eb1aefb9 in ?? ()
3 LWP 9862 "kudu" 0x00007fd0eb1aefb9 in ?? ()
4 LWP 9863 "kudu" 0x00007fd0eb1aefb9 in ?? ()
5 LWP 9864 "kernel-watcher-" 0x00007fd0eb1aefb9 in ?? ()
6 LWP 9870 "ntp client-9870" 0x00007fd0eb1b29e2 in ?? ()
7 LWP 9871 "file cache-evic" 0x00007fd0eb1aefb9 in ?? ()
8 LWP 9872 "sq_acceptor" 0x00007fd0e3ccccb9 in ?? ()
9 LWP 9875 "rpc reactor-987" 0x00007fd0e3cd9a47 in ?? ()
10 LWP 9876 "rpc reactor-987" 0x00007fd0e3cd9a47 in ?? ()
11 LWP 9877 "rpc reactor-987" 0x00007fd0e3cd9a47 in ?? ()
12 LWP 9878 "rpc reactor-987" 0x00007fd0e3cd9a47 in ?? ()
13 LWP 9879 "MaintenanceMgr " 0x00007fd0eb1aead3 in ?? ()
14 LWP 9880 "txn-status-mana" 0x00007fd0eb1aefb9 in ?? ()
15 LWP 9882 "collect_and_rem" 0x00007fd0eb1aefb9 in ?? ()
16 LWP 9883 "tc-session-exp-" 0x00007fd0eb1aefb9 in ?? ()
17 LWP 9884 "rpc worker-9884" 0x00007fd0eb1aead3 in ?? ()
18 LWP 9885 "rpc worker-9885" 0x00007fd0eb1aead3 in ?? ()
19 LWP 9886 "rpc worker-9886" 0x00007fd0eb1aead3 in ?? ()
20 LWP 9887 "rpc worker-9887" 0x00007fd0eb1aead3 in ?? ()
21 LWP 9888 "rpc worker-9888" 0x00007fd0eb1aead3 in ?? ()
22 LWP 9889 "rpc worker-9889" 0x00007fd0eb1aead3 in ?? ()
23 LWP 9890 "rpc worker-9890" 0x00007fd0eb1aead3 in ?? ()
24 LWP 9891 "rpc worker-9891" 0x00007fd0eb1aead3 in ?? ()
25 LWP 9892 "rpc worker-9892" 0x00007fd0eb1aead3 in ?? ()
26 LWP 9893 "rpc worker-9893" 0x00007fd0eb1aead3 in ?? ()
27 LWP 9894 "rpc worker-9894" 0x00007fd0eb1aead3 in ?? ()
28 LWP 9895 "rpc worker-9895" 0x00007fd0eb1aead3 in ?? ()
29 LWP 9896 "rpc worker-9896" 0x00007fd0eb1aead3 in ?? ()
30 LWP 9897 "rpc worker-9897" 0x00007fd0eb1aead3 in ?? ()
31 LWP 9898 "rpc worker-9898" 0x00007fd0eb1aead3 in ?? ()
32 LWP 9899 "rpc worker-9899" 0x00007fd0eb1aead3 in ?? ()
33 LWP 9900 "rpc worker-9900" 0x00007fd0eb1aead3 in ?? ()
34 LWP 9901 "rpc worker-9901" 0x00007fd0eb1aead3 in ?? ()
35 LWP 9902 "rpc worker-9902" 0x00007fd0eb1aead3 in ?? ()
36 LWP 9903 "rpc worker-9903" 0x00007fd0eb1aead3 in ?? ()
37 LWP 9904 "rpc worker-9904" 0x00007fd0eb1aead3 in ?? ()
38 LWP 9905 "rpc worker-9905" 0x00007fd0eb1aead3 in ?? ()
39 LWP 9906 "rpc worker-9906" 0x00007fd0eb1aead3 in ?? ()
40 LWP 9907 "rpc worker-9907" 0x00007fd0eb1aead3 in ?? ()
41 LWP 9908 "rpc worker-9908" 0x00007fd0eb1aead3 in ?? ()
42 LWP 9909 "rpc worker-9909" 0x00007fd0eb1aead3 in ?? ()
43 LWP 9910 "rpc worker-9910" 0x00007fd0eb1aead3 in ?? ()
44 LWP 9911 "rpc worker-9911" 0x00007fd0eb1aead3 in ?? ()
45 LWP 9912 "rpc worker-9912" 0x00007fd0eb1aead3 in ?? ()
46 LWP 9913 "rpc worker-9913" 0x00007fd0eb1aead3 in ?? ()
47 LWP 9914 "rpc worker-9914" 0x00007fd0eb1aead3 in ?? ()
48 LWP 9915 "rpc worker-9915" 0x00007fd0eb1aead3 in ?? ()
49 LWP 9916 "rpc worker-9916" 0x00007fd0eb1aead3 in ?? ()
50 LWP 9917 "rpc worker-9917" 0x00007fd0eb1aead3 in ?? ()
51 LWP 9918 "rpc worker-9918" 0x00007fd0eb1aead3 in ?? ()
52 LWP 9919 "rpc worker-9919" 0x00007fd0eb1aead3 in ?? ()
53 LWP 9920 "rpc worker-9920" 0x00007fd0eb1aead3 in ?? ()
54 LWP 9921 "rpc worker-9921" 0x00007fd0eb1aead3 in ?? ()
55 LWP 9922 "rpc worker-9922" 0x00007fd0eb1aead3 in ?? ()
56 LWP 9923 "rpc worker-9923" 0x00007fd0eb1aead3 in ?? ()
57 LWP 9924 "rpc worker-9924" 0x00007fd0eb1aead3 in ?? ()
58 LWP 9925 "rpc worker-9925" 0x00007fd0eb1aead3 in ?? ()
59 LWP 9926 "rpc worker-9926" 0x00007fd0eb1aead3 in ?? ()
60 LWP 9927 "rpc worker-9927" 0x00007fd0eb1aead3 in ?? ()
61 LWP 9928 "rpc worker-9928" 0x00007fd0eb1aead3 in ?? ()
62 LWP 9929 "rpc worker-9929" 0x00007fd0eb1aead3 in ?? ()
63 LWP 9930 "rpc worker-9930" 0x00007fd0eb1aead3 in ?? ()
64 LWP 9931 "rpc worker-9931" 0x00007fd0eb1aead3 in ?? ()
65 LWP 9932 "rpc worker-9932" 0x00007fd0eb1aead3 in ?? ()
66 LWP 9933 "rpc worker-9933" 0x00007fd0eb1aead3 in ?? ()
67 LWP 9934 "rpc worker-9934" 0x00007fd0eb1aead3 in ?? ()
68 LWP 9935 "rpc worker-9935" 0x00007fd0eb1aead3 in ?? ()
69 LWP 9936 "rpc worker-9936" 0x00007fd0eb1aead3 in ?? ()
70 LWP 9937 "rpc worker-9937" 0x00007fd0eb1aead3 in ?? ()
71 LWP 9938 "rpc worker-9938" 0x00007fd0eb1aead3 in ?? ()
72 LWP 9939 "rpc worker-9939" 0x00007fd0eb1aead3 in ?? ()
73 LWP 9940 "rpc worker-9940" 0x00007fd0eb1aead3 in ?? ()
74 LWP 9941 "rpc worker-9941" 0x00007fd0eb1aead3 in ?? ()
75 LWP 9942 "rpc worker-9942" 0x00007fd0eb1aead3 in ?? ()
76 LWP 9943 "rpc worker-9943" 0x00007fd0eb1aead3 in ?? ()
77 LWP 9944 "rpc worker-9944" 0x00007fd0eb1aead3 in ?? ()
78 LWP 9945 "rpc worker-9945" 0x00007fd0eb1aead3 in ?? ()
79 LWP 9946 "rpc worker-9946" 0x00007fd0eb1aead3 in ?? ()
80 LWP 9947 "rpc worker-9947" 0x00007fd0eb1aead3 in ?? ()
81 LWP 9948 "rpc worker-9948" 0x00007fd0eb1aead3 in ?? ()
82 LWP 9949 "rpc worker-9949" 0x00007fd0eb1aead3 in ?? ()
83 LWP 9950 "rpc worker-9950" 0x00007fd0eb1aead3 in ?? ()
84 LWP 9951 "rpc worker-9951" 0x00007fd0eb1aead3 in ?? ()
85 LWP 9952 "rpc worker-9952" 0x00007fd0eb1aead3 in ?? ()
86 LWP 9953 "rpc worker-9953" 0x00007fd0eb1aead3 in ?? ()
87 LWP 9954 "rpc worker-9954" 0x00007fd0eb1aead3 in ?? ()
88 LWP 9955 "rpc worker-9955" 0x00007fd0eb1aead3 in ?? ()
89 LWP 9956 "rpc worker-9956" 0x00007fd0eb1aead3 in ?? ()
90 LWP 9957 "rpc worker-9957" 0x00007fd0eb1aead3 in ?? ()
91 LWP 9958 "rpc worker-9958" 0x00007fd0eb1aead3 in ?? ()
92 LWP 9959 "rpc worker-9959" 0x00007fd0eb1aead3 in ?? ()
93 LWP 9960 "rpc worker-9960" 0x00007fd0eb1aead3 in ?? ()
94 LWP 9961 "rpc worker-9961" 0x00007fd0eb1aead3 in ?? ()
95 LWP 9962 "rpc worker-9962" 0x00007fd0eb1aead3 in ?? ()
96 LWP 9963 "rpc worker-9963" 0x00007fd0eb1aead3 in ?? ()
97 LWP 9964 "rpc worker-9964" 0x00007fd0eb1aead3 in ?? ()
98 LWP 9965 "rpc worker-9965" 0x00007fd0eb1aead3 in ?? ()
99 LWP 9966 "rpc worker-9966" 0x00007fd0eb1aead3 in ?? ()
100 LWP 9967 "rpc worker-9967" 0x00007fd0eb1aead3 in ?? ()
101 LWP 9968 "rpc worker-9968" 0x00007fd0eb1aead3 in ?? ()
102 LWP 9969 "rpc worker-9969" 0x00007fd0eb1aead3 in ?? ()
103 LWP 9970 "rpc worker-9970" 0x00007fd0eb1aead3 in ?? ()
104 LWP 9971 "rpc worker-9971" 0x00007fd0eb1aead3 in ?? ()
105 LWP 9972 "rpc worker-9972" 0x00007fd0eb1aead3 in ?? ()
106 LWP 9973 "rpc worker-9973" 0x00007fd0eb1aead3 in ?? ()
107 LWP 9974 "rpc worker-9974" 0x00007fd0eb1aead3 in ?? ()
108 LWP 9975 "rpc worker-9975" 0x00007fd0eb1aead3 in ?? ()
109 LWP 9976 "rpc worker-9976" 0x00007fd0eb1aead3 in ?? ()
110 LWP 9977 "rpc worker-9977" 0x00007fd0eb1aead3 in ?? ()
111 LWP 9978 "rpc worker-9978" 0x00007fd0eb1aead3 in ?? ()
112 LWP 9979 "rpc worker-9979" 0x00007fd0eb1aead3 in ?? ()
113 LWP 9980 "rpc worker-9980" 0x00007fd0eb1aead3 in ?? ()
114 LWP 9981 "rpc worker-9981" 0x00007fd0eb1aead3 in ?? ()
115 LWP 9982 "rpc worker-9982" 0x00007fd0eb1aead3 in ?? ()
116 LWP 9983 "rpc worker-9983" 0x00007fd0eb1aead3 in ?? ()
117 LWP 9984 "diag-logger-998" 0x00007fd0eb1aefb9 in ?? ()
118 LWP 9985 "result-tracker-" 0x00007fd0eb1aefb9 in ?? ()
119 LWP 9986 "excess-log-dele" 0x00007fd0eb1aefb9 in ?? ()
120 LWP 9987 "acceptor-9987" 0x00007fd0e3cdb0c7 in ?? ()
121 LWP 9988 "heartbeat-9988" 0x00007fd0eb1aefb9 in ?? ()
122 LWP 9989 "maintenance_sch" 0x00007fd0eb1aefb9 in ?? ()
123 LWP 10314 "wal-append [wor" 0x00007fd0eb1aefb9 in ?? ()
Thread 123 (LWP 10314):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x00006020002ec9f8 in ?? ()
#2 0x0000000000000390 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061a0003a0fb0 in ?? ()
#5 0x00007fd09e790130 in ?? ()
#6 0x0000000000000720 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 122 (LWP 9989):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x00007fd09f7d0700 in ?? ()
#2 0x000000000000003a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000016ca0 in ?? ()
#5 0x00007fd09f7d0750 in ?? ()
#6 0x0000000000000074 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 9988):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0xffe6cd9ab650d2a9 in ?? ()
#2 0x0000000000000010 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x000061300001a904 in ?? ()
#5 0x00007fd09ffe8610 in ?? ()
#6 0x0000000000000021 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007fd09ffe8630 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007fd09ffe86b0 in ?? ()
#12 0x00000ffa13ffd0d8 in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 9987):
#0 0x00007fd0e3cdb0c7 in ?? ()
#1 0x00006140000210a8 in ?? ()
#2 0x00007fd0a080fd70 in ?? ()
#3 0x00007fd0a080fda0 in ?? ()
#4 0x00007fd0a080fea0 in ?? ()
#5 0x00007fd0a080fd90 in ?? ()
#6 0x00007fd0a080fe00 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008c485b in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007fd0a080ff20 in ?? ()
#11 0x00007fd0a080f65c in ?? ()
#12 0x00000032a080f5d0 in ?? ()
#13 0x00007fd0a0013000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 9986):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x00007fd0a1028f60 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 9985):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x00007fd0a1842120 in ?? ()
#2 0x000000000000000e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061100008abb0 in ?? ()
#5 0x00007fd0a1842110 in ?? ()
#6 0x000000000000001c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 9984):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 9983):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000007 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b684c in ?? ()
#4 0x00007fd0a2871eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fd0a2871ed0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b6800 in ?? ()
#9 0x00007fd0eb1ae770 in ?? ()
#10 0x00007fd0a2871ed0 in ?? ()
#11 0x00007fd0a2871e90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 9982):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007fd0a3089eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fd0a3089ed0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007fd0eb1ae770 in ?? ()
#10 0x00007fd0a3089ed0 in ?? ()
#11 0x00007fd0a3089e90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 9981):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 9980):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 9979):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 9978):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 9977):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 9976):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 9975):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 9974):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 9973):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 9972):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 9971):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 9970):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 9969):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 9968):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 9967):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 9966):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 9965):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 9964):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 9963):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 9962):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 9961):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 9960):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 9959):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 9958):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 9957):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 9956):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 9955):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 9954):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 9953):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 9952):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 9951):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 9950):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 9949):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 9948):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 9947):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 9946):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 9945):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 9944):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 9943):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x00000000000001c6 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007fd0b6c32eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fd0b6c32ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 9942):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x000000000000013e in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007fd0b744aeb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fd0b744aed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 9941):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 9940):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 9939):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 9938):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 9937):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 9936):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 9935):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 9934):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 9933):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 9932):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 9931):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 9930):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 9929):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 9928):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 9927):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 9926):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 9925):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 9924):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 9923):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007fd0c0e13eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fd0c0e13ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 9922):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 9921):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 9920):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 9919):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 9918):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 9917):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 9916):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 9915):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 9914):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 9913):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 9912):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 9911):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 9910):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 9909):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 9908):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 9907):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 9906):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 9905):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 9904):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 9903):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x000000000000005f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007fd0caff4eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fd0caff4ed0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007fd0eb1ae770 in ?? ()
#10 0x00007fd0caff4ed0 in ?? ()
#11 0x00007fd0caff4e90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 9902):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007fd0cb80ceb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fd0cb80ced0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007fd0eb1ae770 in ?? ()
#10 0x00007fd0cb80ced0 in ?? ()
#11 0x00007fd0cb80ce90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 34 (LWP 9901):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 9900):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 9899):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 9898):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 9897):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 9896):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 9895):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 9894):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 9893):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 9892):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 9891):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 9890):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 9889):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 9888):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 9887):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 9886):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 9885):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 9884):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 9883):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x00007fd0d5204ce0 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b8a0 in ?? ()
#5 0x00007fd0d5204cd0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 9882):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x0000000000003ff8 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001df18 in ?? ()
#5 0x00007fd0d5a3c270 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 14 (LWP 9880):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x00007fd0d6a99260 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000c5a8 in ?? ()
#5 0x00007fd0d6a99250 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 9879):
#0 0x00007fd0eb1aead3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 9878):
#0 0x00007fd0e3cd9a47 in ?? ()
#1 0x00007fd0d7ab2340 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007fd0d7ab2330 in ?? ()
#4 0x00007fd0d7ab2540 in ?? ()
#5 0x00007fd0d7ab2380 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007fd0d7ab2400 in ?? ()
#8 0x00007fd0e634c25d in ?? ()
#9 0x3fb98ca35a191000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x00000000d8000000 in ?? ()
#14 0x00007fd000000000 in ?? ()
#15 0x41da4ecba51ccc95 in ?? ()
#16 0x00000ffa9af4e480 in ?? ()
#17 0x00007fd0d7ab23e0 in ?? ()
#18 0x00007fd0e6350ba3 in ?? ()
#19 0x00007fd0d7ab23b0 in ?? ()
#20 0x3fb98ca35a191000 in ?? ()
#21 0x00000000d7ab2400 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb98ca35a191000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 9877):
#0 0x00007fd0e3cd9a47 in ?? ()
#1 0x00007fd0d82b3340 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007fd0d82b3330 in ?? ()
#4 0x00007fd0d82b3540 in ?? ()
#5 0x00007fd0d82b3380 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007fd0d82b3400 in ?? ()
#8 0x00007fd0e634c25d in ?? ()
#9 0x3fb98b903d395000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000c9800000 in ?? ()
#14 0x00007fd000000000 in ?? ()
#15 0x41da4ecba51ccc92 in ?? ()
#16 0x00000ffa9b04e680 in ?? ()
#17 0x00007fd0d82b33e0 in ?? ()
#18 0x00007fd0e6350ba3 in ?? ()
#19 0x00007fd0d82b33b0 in ?? ()
#20 0x3fb98b903d395000 in ?? ()
#21 0x00000000d82b3400 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb98b903d395000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 9876):
#0 0x00007fd0e3cd9a47 in ?? ()
#1 0x00007fd0d8ab4340 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007fd0d8ab4330 in ?? ()
#4 0x00007fd0d8ab4540 in ?? ()
#5 0x00007fd0d8ab4380 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007fd0d8ab4400 in ?? ()
#8 0x00007fd0e634c25d in ?? ()
#9 0x3fb2338edeea7000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x0000000079400000 in ?? ()
#14 0x00007fd000000000 in ?? ()
#15 0x41da4ecba51ccc91 in ?? ()
#16 0x00000ffa9b14e880 in ?? ()
#17 0x00007fd0d8ab43e0 in ?? ()
#18 0x00007fd0e6350ba3 in ?? ()
#19 0x00007fd0d8ab43b0 in ?? ()
#20 0x3fb2338edeea7000 in ?? ()
#21 0x00000000d8ab4400 in ?? ()
#22 0x000061a00000ba80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb2338edeea7000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 9875):
#0 0x00007fd0e3cd9a47 in ?? ()
#1 0x00007fd0daaad340 in ?? ()
#2 0x000061a00000b480 in ?? ()
#3 0x00007fd0daaad330 in ?? ()
#4 0x00007fd0daaad540 in ?? ()
#5 0x00007fd0daaad380 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007fd0daaad400 in ?? ()
#8 0x00007fd0e634c25d in ?? ()
#9 0x3fb97ca017437000 in ?? ()
#10 0x000061a00000b480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000b480 in ?? ()
#13 0x0000000033000000 in ?? ()
#14 0x00007fd000000000 in ?? ()
#15 0x41da4ecba51ccc93 in ?? ()
#16 0x00000ffa9b54da80 in ?? ()
#17 0x00007fd0daaad3e0 in ?? ()
#18 0x00007fd0e6350ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 9872):
#0 0x00007fd0e3ccccb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007fd0dc4b77b8 in ?? ()
#3 0x000060200001d7d0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008bc4a1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 9871):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x00007fd0dbcb60e0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 9870):
#0 0x00007fd0eb1b29e2 in ?? ()
#1 0x00007fd0db4b3bc0 in ?? ()
#2 0x00007fd0db4b3c60 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007fd0db4b3c10 in ?? ()
#6 0x00000000017ab3a0 in ?? ()
#7 0x00007fd0db4b3c70 in ?? ()
#8 0x000061100008b510 in ?? ()
#9 0x00007fd0db4b3c60 in ?? ()
#10 0x00000000008b6997 in __sanitizer::theDepot ()
#11 0x00007fd0f0adc61c in ?? ()
#12 0x00007fd0f0acbc29 in ?? ()
#13 0x00007fd0f0ad0216 in ?? ()
#14 0x00007fd0f0ad4c50 in ?? ()
#15 0x00007fd0f0ad4a79 in ?? ()
#16 0x0000000000a8970d in __sanitizer::theDepot ()
#17 0x00007fd0e7b9d3f9 in ?? ()
#18 0x00007fd0eb1a86db in ?? ()
#19 0x00000ffa9b68e688 in ?? ()
#20 0x00007fd0db4b3460 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 9864):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x00007fd0dd4b8ca0 in ?? ()
#2 0x0000000000000049 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001e728 in ?? ()
#5 0x00007fd0dd4b8c90 in ?? ()
#6 0x0000000000000092 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 9863):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 9862):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 9861):
#0 0x00007fd0eb1aefb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007fd0decbc450 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 9859):
#0 0x00007fd0eb1b2d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20251211 20:56:34.048259 5967 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 1 with UUID 5ccd26b0e8fe409f91fbd1da24bc7137 and pid 9722
************************ BEGIN STACKS **************************
[New LWP 9724]
[New LWP 9725]
[New LWP 9726]
[New LWP 9727]
[New LWP 9733]
[New LWP 9734]
[New LWP 9735]
[New LWP 9738]
[New LWP 9739]
[New LWP 9740]
[New LWP 9741]
[New LWP 9742]
[New LWP 9743]
[New LWP 9745]
[New LWP 9746]
[New LWP 9747]
[New LWP 9748]
[New LWP 9749]
[New LWP 9750]
[New LWP 9751]
[New LWP 9752]
[New LWP 9753]
[New LWP 9754]
[New LWP 9755]
[New LWP 9756]
[New LWP 9757]
[New LWP 9758]
[New LWP 9759]
[New LWP 9760]
[New LWP 9761]
[New LWP 9762]
[New LWP 9763]
[New LWP 9764]
[New LWP 9765]
[New LWP 9766]
[New LWP 9767]
[New LWP 9768]
[New LWP 9769]
[New LWP 9770]
[New LWP 9771]
[New LWP 9772]
[New LWP 9773]
[New LWP 9774]
[New LWP 9775]
[New LWP 9776]
[New LWP 9777]
[New LWP 9778]
[New LWP 9779]
[New LWP 9780]
[New LWP 9781]
[New LWP 9782]
[New LWP 9783]
[New LWP 9784]
[New LWP 9785]
[New LWP 9786]
[New LWP 9787]
[New LWP 9788]
[New LWP 9789]
[New LWP 9790]
[New LWP 9791]
[New LWP 9792]
[New LWP 9793]
[New LWP 9794]
[New LWP 9795]
[New LWP 9796]
[New LWP 9797]
[New LWP 9798]
[New LWP 9799]
[New LWP 9800]
[New LWP 9801]
[New LWP 9802]
[New LWP 9803]
[New LWP 9804]
[New LWP 9805]
[New LWP 9806]
[New LWP 9807]
[New LWP 9808]
[New LWP 9809]
[New LWP 9810]
[New LWP 9811]
[New LWP 9812]
[New LWP 9813]
[New LWP 9814]
[New LWP 9815]
[New LWP 9816]
[New LWP 9817]
[New LWP 9818]
[New LWP 9819]
[New LWP 9820]
[New LWP 9821]
[New LWP 9822]
[New LWP 9823]
[New LWP 9824]
[New LWP 9825]
[New LWP 9826]
[New LWP 9827]
[New LWP 9828]
[New LWP 9829]
[New LWP 9830]
[New LWP 9831]
[New LWP 9832]
[New LWP 9833]
[New LWP 9834]
[New LWP 9835]
[New LWP 9836]
[New LWP 9837]
[New LWP 9838]
[New LWP 9839]
[New LWP 9840]
[New LWP 9841]
[New LWP 9842]
[New LWP 9843]
[New LWP 9844]
[New LWP 9845]
[New LWP 9846]
[New LWP 9847]
[New LWP 9848]
[New LWP 9849]
[New LWP 9850]
[New LWP 9851]
[New LWP 9852]
[New LWP 10308]
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c72
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c72
0x00007f6103d38d50 in ?? ()
Id Target Id Frame
* 1 LWP 9722 "kudu" 0x00007f6103d38d50 in ?? ()
2 LWP 9724 "kudu" 0x00007f6103d34fb9 in ?? ()
3 LWP 9725 "kudu" 0x00007f6103d34fb9 in ?? ()
4 LWP 9726 "kudu" 0x00007f6103d34fb9 in ?? ()
5 LWP 9727 "kernel-watcher-" 0x00007f6103d34fb9 in ?? ()
6 LWP 9733 "ntp client-9733" 0x00007f6103d389e2 in ?? ()
7 LWP 9734 "file cache-evic" 0x00007f6103d34fb9 in ?? ()
8 LWP 9735 "sq_acceptor" 0x00007f60fc852cb9 in ?? ()
9 LWP 9738 "rpc reactor-973" 0x00007f60fc85fa47 in ?? ()
10 LWP 9739 "rpc reactor-973" 0x00007f60fc85fa47 in ?? ()
11 LWP 9740 "rpc reactor-974" 0x00007f60fc85fa47 in ?? ()
12 LWP 9741 "rpc reactor-974" 0x00007f60fc85fa47 in ?? ()
13 LWP 9742 "MaintenanceMgr " 0x00007f6103d34ad3 in ?? ()
14 LWP 9743 "txn-status-mana" 0x00007f6103d34fb9 in ?? ()
15 LWP 9745 "collect_and_rem" 0x00007f6103d34fb9 in ?? ()
16 LWP 9746 "tc-session-exp-" 0x00007f6103d34fb9 in ?? ()
17 LWP 9747 "rpc worker-9747" 0x00007f6103d34ad3 in ?? ()
18 LWP 9748 "rpc worker-9748" 0x00007f6103d34ad3 in ?? ()
19 LWP 9749 "rpc worker-9749" 0x00007f6103d34ad3 in ?? ()
20 LWP 9750 "rpc worker-9750" 0x00007f6103d34ad3 in ?? ()
21 LWP 9751 "rpc worker-9751" 0x00007f6103d34ad3 in ?? ()
22 LWP 9752 "rpc worker-9752" 0x00007f6103d34ad3 in ?? ()
23 LWP 9753 "rpc worker-9753" 0x00007f6103d34ad3 in ?? ()
24 LWP 9754 "rpc worker-9754" 0x00007f6103d34ad3 in ?? ()
25 LWP 9755 "rpc worker-9755" 0x00007f6103d34ad3 in ?? ()
26 LWP 9756 "rpc worker-9756" 0x00007f6103d34ad3 in ?? ()
27 LWP 9757 "rpc worker-9757" 0x00007f6103d34ad3 in ?? ()
28 LWP 9758 "rpc worker-9758" 0x00007f6103d34ad3 in ?? ()
29 LWP 9759 "rpc worker-9759" 0x00007f6103d34ad3 in ?? ()
30 LWP 9760 "rpc worker-9760" 0x00007f6103d34ad3 in ?? ()
31 LWP 9761 "rpc worker-9761" 0x00007f6103d34ad3 in ?? ()
32 LWP 9762 "rpc worker-9762" 0x00007f6103d34ad3 in ?? ()
33 LWP 9763 "rpc worker-9763" 0x00007f6103d34ad3 in ?? ()
34 LWP 9764 "rpc worker-9764" 0x00007f6103d34ad3 in ?? ()
35 LWP 9765 "rpc worker-9765" 0x00007f6103d34ad3 in ?? ()
36 LWP 9766 "rpc worker-9766" 0x00007f6103d34ad3 in ?? ()
37 LWP 9767 "rpc worker-9767" 0x00007f6103d34ad3 in ?? ()
38 LWP 9768 "rpc worker-9768" 0x00007f6103d34ad3 in ?? ()
39 LWP 9769 "rpc worker-9769" 0x00007f6103d34ad3 in ?? ()
40 LWP 9770 "rpc worker-9770" 0x00007f6103d34ad3 in ?? ()
41 LWP 9771 "rpc worker-9771" 0x00007f6103d34ad3 in ?? ()
42 LWP 9772 "rpc worker-9772" 0x00007f6103d34ad3 in ?? ()
43 LWP 9773 "rpc worker-9773" 0x00007f6103d34ad3 in ?? ()
44 LWP 9774 "rpc worker-9774" 0x00007f6103d34ad3 in ?? ()
45 LWP 9775 "rpc worker-9775" 0x00007f6103d34ad3 in ?? ()
46 LWP 9776 "rpc worker-9776" 0x00007f6103d34ad3 in ?? ()
47 LWP 9777 "rpc worker-9777" 0x00007f6103d34ad3 in ?? ()
48 LWP 9778 "rpc worker-9778" 0x00007f6103d34ad3 in ?? ()
49 LWP 9779 "rpc worker-9779" 0x00007f6103d34ad3 in ?? ()
50 LWP 9780 "rpc worker-9780" 0x00007f6103d34ad3 in ?? ()
51 LWP 9781 "rpc worker-9781" 0x00007f6103d34ad3 in ?? ()
52 LWP 9782 "rpc worker-9782" 0x00007f6103d34ad3 in ?? ()
53 LWP 9783 "rpc worker-9783" 0x00007f6103d34ad3 in ?? ()
54 LWP 9784 "rpc worker-9784" 0x00007f6103d34ad3 in ?? ()
55 LWP 9785 "rpc worker-9785" 0x00007f6103d34ad3 in ?? ()
56 LWP 9786 "rpc worker-9786" 0x00007f6103d34ad3 in ?? ()
57 LWP 9787 "rpc worker-9787" 0x00007f6103d34ad3 in ?? ()
58 LWP 9788 "rpc worker-9788" 0x00007f6103d34ad3 in ?? ()
59 LWP 9789 "rpc worker-9789" 0x00007f6103d34ad3 in ?? ()
60 LWP 9790 "rpc worker-9790" 0x00007f6103d34ad3 in ?? ()
61 LWP 9791 "rpc worker-9791" 0x00007f6103d34ad3 in ?? ()
62 LWP 9792 "rpc worker-9792" 0x00007f6103d34ad3 in ?? ()
63 LWP 9793 "rpc worker-9793" 0x00007f6103d34ad3 in ?? ()
64 LWP 9794 "rpc worker-9794" 0x00007f6103d34ad3 in ?? ()
65 LWP 9795 "rpc worker-9795" 0x00007f6103d34ad3 in ?? ()
66 LWP 9796 "rpc worker-9796" 0x00007f6103d34ad3 in ?? ()
67 LWP 9797 "rpc worker-9797" 0x00007f6103d34ad3 in ?? ()
68 LWP 9798 "rpc worker-9798" 0x00007f6103d34ad3 in ?? ()
69 LWP 9799 "rpc worker-9799" 0x00007f6103d34ad3 in ?? ()
70 LWP 9800 "rpc worker-9800" 0x00007f6103d34ad3 in ?? ()
71 LWP 9801 "rpc worker-9801" 0x00007f6103d34ad3 in ?? ()
72 LWP 9802 "rpc worker-9802" 0x00007f6103d34ad3 in ?? ()
73 LWP 9803 "rpc worker-9803" 0x00007f6103d34ad3 in ?? ()
74 LWP 9804 "rpc worker-9804" 0x00007f6103d34ad3 in ?? ()
75 LWP 9805 "rpc worker-9805" 0x00007f6103d34ad3 in ?? ()
76 LWP 9806 "rpc worker-9806" 0x00007f6103d34ad3 in ?? ()
77 LWP 9807 "rpc worker-9807" 0x00007f6103d34ad3 in ?? ()
78 LWP 9808 "rpc worker-9808" 0x00007f6103d34ad3 in ?? ()
79 LWP 9809 "rpc worker-9809" 0x00007f6103d34ad3 in ?? ()
80 LWP 9810 "rpc worker-9810" 0x00007f6103d34ad3 in ?? ()
81 LWP 9811 "rpc worker-9811" 0x00007f6103d34ad3 in ?? ()
82 LWP 9812 "rpc worker-9812" 0x00007f6103d34ad3 in ?? ()
83 LWP 9813 "rpc worker-9813" 0x00007f6103d34ad3 in ?? ()
84 LWP 9814 "rpc worker-9814" 0x00007f6103d34ad3 in ?? ()
85 LWP 9815 "rpc worker-9815" 0x00007f6103d34ad3 in ?? ()
86 LWP 9816 "rpc worker-9816" 0x00007f6103d34ad3 in ?? ()
87 LWP 9817 "rpc worker-9817" 0x00007f6103d34ad3 in ?? ()
88 LWP 9818 "rpc worker-9818" 0x00007f6103d34ad3 in ?? ()
89 LWP 9819 "rpc worker-9819" 0x00007f6103d34ad3 in ?? ()
90 LWP 9820 "rpc worker-9820" 0x00007f6103d34ad3 in ?? ()
91 LWP 9821 "rpc worker-9821" 0x00007f6103d34ad3 in ?? ()
92 LWP 9822 "rpc worker-9822" 0x00007f6103d34ad3 in ?? ()
93 LWP 9823 "rpc worker-9823" 0x00007f6103d34ad3 in ?? ()
94 LWP 9824 "rpc worker-9824" 0x00007f6103d34ad3 in ?? ()
95 LWP 9825 "rpc worker-9825" 0x00007f6103d34ad3 in ?? ()
96 LWP 9826 "rpc worker-9826" 0x00007f6103d34ad3 in ?? ()
97 LWP 9827 "rpc worker-9827" 0x00007f6103d34ad3 in ?? ()
98 LWP 9828 "rpc worker-9828" 0x00007f6103d34ad3 in ?? ()
99 LWP 9829 "rpc worker-9829" 0x00007f6103d34ad3 in ?? ()
100 LWP 9830 "rpc worker-9830" 0x00007f6103d34ad3 in ?? ()
101 LWP 9831 "rpc worker-9831" 0x00007f6103d34ad3 in ?? ()
102 LWP 9832 "rpc worker-9832" 0x00007f6103d34ad3 in ?? ()
103 LWP 9833 "rpc worker-9833" 0x00007f6103d34ad3 in ?? ()
104 LWP 9834 "rpc worker-9834" 0x00007f6103d34ad3 in ?? ()
105 LWP 9835 "rpc worker-9835" 0x00007f6103d34ad3 in ?? ()
106 LWP 9836 "rpc worker-9836" 0x00007f6103d34ad3 in ?? ()
107 LWP 9837 "rpc worker-9837" 0x00007f6103d34ad3 in ?? ()
108 LWP 9838 "rpc worker-9838" 0x00007f6103d34ad3 in ?? ()
109 LWP 9839 "rpc worker-9839" 0x00007f6103d34ad3 in ?? ()
110 LWP 9840 "rpc worker-9840" 0x00007f6103d34ad3 in ?? ()
111 LWP 9841 "rpc worker-9841" 0x00007f6103d34ad3 in ?? ()
112 LWP 9842 "rpc worker-9842" 0x00007f6103d34ad3 in ?? ()
113 LWP 9843 "rpc worker-9843" 0x00007f6103d34ad3 in ?? ()
114 LWP 9844 "rpc worker-9844" 0x00007f6103d34ad3 in ?? ()
115 LWP 9845 "rpc worker-9845" 0x00007f6103d34ad3 in ?? ()
116 LWP 9846 "rpc worker-9846" 0x00007f6103d34ad3 in ?? ()
117 LWP 9847 "diag-logger-984" 0x00007f6103d34fb9 in ?? ()
118 LWP 9848 "result-tracker-" 0x00007f6103d34fb9 in ?? ()
119 LWP 9849 "excess-log-dele" 0x00007f6103d34fb9 in ?? ()
120 LWP 9850 "acceptor-9850" 0x00007f60fc8610c7 in ?? ()
121 LWP 9851 "heartbeat-9851" 0x00007f6103d34fb9 in ?? ()
122 LWP 9852 "maintenance_sch" 0x00007f6103d34fb9 in ?? ()
123 LWP 10308 "raft [worker]-1" 0x00007f6103d34fb9 in ?? ()
Thread 123 (LWP 10308):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000333 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007f60eee0fbc4 in ?? ()
#5 0x00007f60eee0f850 in ?? ()
#6 0x0000000000000667 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f60eee0f870 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 122 (LWP 9852):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x00007f60b8356700 in ?? ()
#2 0x0000000000000040 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000016ca0 in ?? ()
#5 0x00007f60b8356750 in ?? ()
#6 0x0000000000000080 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 9851):
#0 0x00007f6103d34fb9 in ?? ()
#1 0xffe6cd9ab650d2a9 in ?? ()
#2 0x0000000000000012 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001a900 in ?? ()
#5 0x00007f60b8b6e610 in ?? ()
#6 0x0000000000000024 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 9850):
#0 0x00007f60fc8610c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007f60b9395d70 in ?? ()
#3 0x00007f60b9395da0 in ?? ()
#4 0x00007f60b9395ea0 in ?? ()
#5 0x00007f60b9395d90 in ?? ()
#6 0x00007f60b9395e00 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008c485b in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007f60b9395f20 in ?? ()
#11 0x00007f60b939565c in ?? ()
#12 0x00000032b93955d0 in ?? ()
#13 0x00007f60b8b99000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 9849):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x00007f60b9baef60 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 9848):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x00007f60ba3c8120 in ?? ()
#2 0x0000000000000010 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061100008abb0 in ?? ()
#5 0x00007f60ba3c8110 in ?? ()
#6 0x0000000000000020 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 9847):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 9846):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000008 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0002b6848 in ?? ()
#4 0x00007f60bb3f7eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f60bb3f7ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 115 (LWP 9845):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 114 (LWP 9844):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 9843):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 9842):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 9841):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 9840):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 9839):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 9838):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 9837):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 9836):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 9835):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 9834):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 9833):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 9832):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 9831):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 9830):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 9829):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 9828):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 9827):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 9826):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 9825):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 9824):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 9823):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 9822):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 9821):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 9820):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 9819):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 9818):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 9817):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 9816):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 9815):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 9814):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 9813):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 9812):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 9811):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 9810):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 9809):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 9808):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 9807):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 9806):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007f60cf7b8eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f60cf7b8ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 9805):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 9804):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 9803):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 9802):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 9801):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 9800):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 9799):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 9798):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 9797):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 9796):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 9795):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 9794):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 9793):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 9792):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 9791):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 9790):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 9789):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 9788):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 9787):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 9786):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007f60d9999eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f60d9999ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 9785):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 9784):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 9783):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 9782):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 9781):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 9780):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 9779):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 9778):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 9777):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 9776):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 9775):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 9774):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 9773):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 9772):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 9771):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 9770):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 9769):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 9768):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 9767):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 9766):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x000000000000018b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f60e3b7aeb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f60e3b7aed0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f6103d34770 in ?? ()
#10 0x00007f60e3b7aed0 in ?? ()
#11 0x00007f60e3b7ae90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 9765):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x00000000000000e4 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f60e4392eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f60e4392ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 9764):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 9763):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 9762):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 9761):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 9760):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 9759):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 9758):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 9757):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 9756):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 9755):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 9754):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 9753):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 9752):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 9751):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 9750):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 9749):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 9748):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 9747):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 9746):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x00007f60edd8fce0 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b8a0 in ?? ()
#5 0x00007f60edd8fcd0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 9745):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x0000000000003ff8 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001df18 in ?? ()
#5 0x00007f60ee5c8270 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 14 (LWP 9743):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x00007f60ef627260 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000c5a8 in ?? ()
#5 0x00007f60ef627250 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 9742):
#0 0x00007f6103d34ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 9741):
#0 0x00007f60fc85fa47 in ?? ()
#1 0x00007f60f0668340 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f60f0668330 in ?? ()
#4 0x00007f60f0668540 in ?? ()
#5 0x00007f60f0668380 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f60f0668400 in ?? ()
#8 0x00007f60feed225d in ?? ()
#9 0x3fa03ed9af894000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x00000000a8000000 in ?? ()
#14 0x00007f6000000000 in ?? ()
#15 0x41da4ecba51ccc94 in ?? ()
#16 0x00000fec9e0c5080 in ?? ()
#17 0x00007f60f06683e0 in ?? ()
#18 0x00007f60feed6ba3 in ?? ()
#19 0x00007f60f06683b0 in ?? ()
#20 0x3fa03ed9af894000 in ?? ()
#21 0x00000000f0668400 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fa03ed9af894000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 9740):
#0 0x00007f60fc85fa47 in ?? ()
#1 0x00007f60f0e7f340 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f60f0e7f330 in ?? ()
#4 0x00007f60f0e7f540 in ?? ()
#5 0x00007f60f0e7f380 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f60f0e7f400 in ?? ()
#8 0x00007f60feed225d in ?? ()
#9 0x3fb98c6245ebd000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x00000000a1000000 in ?? ()
#14 0x00007f6000000000 in ?? ()
#15 0x41da4ecba51ccc93 in ?? ()
#16 0x00000fec9e1c7e80 in ?? ()
#17 0x00007f60f0e7f3e0 in ?? ()
#18 0x00007f60feed6ba3 in ?? ()
#19 0x00007f60f0e7f3b0 in ?? ()
#20 0x3fb98c6245ebd000 in ?? ()
#21 0x00000000f0e7f400 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb98c6245ebd000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 9739):
#0 0x00007f60fc85fa47 in ?? ()
#1 0x00007f60f1696340 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f60f1696330 in ?? ()
#4 0x00007f60f1696540 in ?? ()
#5 0x00007f60f1696380 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f60f1696400 in ?? ()
#8 0x00007f60feed225d in ?? ()
#9 0x3f90c6822ff0c000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x000000006c800000 in ?? ()
#14 0x00007f6000000000 in ?? ()
#15 0x41da4ecba51ccc93 in ?? ()
#16 0x00000fec9e2cac80 in ?? ()
#17 0x00007f60f16963e0 in ?? ()
#18 0x00007f60feed6ba3 in ?? ()
#19 0x00007f60f16963b0 in ?? ()
#20 0x3f90c6822ff0c000 in ?? ()
#21 0x00000000f1696400 in ?? ()
#22 0x000061a00000ba80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3f90c6822ff0c000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 9738):
#0 0x00007f60fc85fa47 in ?? ()
#1 0x00007f60f36ad340 in ?? ()
#2 0x000061a00000b480 in ?? ()
#3 0x00007f60f36ad330 in ?? ()
#4 0x00007f60f36ad540 in ?? ()
#5 0x00007f60f36ad380 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f60f36ad400 in ?? ()
#8 0x00007f60feed225d in ?? ()
#9 0x3fb951cf3786b000 in ?? ()
#10 0x000061a00000b480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000b480 in ?? ()
#13 0x00000000d6000000 in ?? ()
#14 0x00007f6000000000 in ?? ()
#15 0x41da4ecba51ccc94 in ?? ()
#16 0x00000fec9e6cda80 in ?? ()
#17 0x00007f60f36ad3e0 in ?? ()
#18 0x00007f60feed6ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 9735):
#0 0x00007f60fc852cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f60f50b77b8 in ?? ()
#3 0x000060200001f790 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008bc4a1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 9734):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x00007f60f48b60e0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 9733):
#0 0x00007f6103d389e2 in ?? ()
#1 0x00007f60f40b3bc0 in ?? ()
#2 0x00007f60f40b3c60 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f60f40b3c10 in ?? ()
#6 0x00000000017ab3a0 in ?? ()
#7 0x00007f60f40b3c70 in ?? ()
#8 0x000061100008b510 in ?? ()
#9 0x00007f60f40b3c60 in ?? ()
#10 0x00000000008b6997 in __sanitizer::theDepot ()
#11 0x00007f610966261c in ?? ()
#12 0x00007f6109651c29 in ?? ()
#13 0x00007f6109656216 in ?? ()
#14 0x00007f610965ac50 in ?? ()
#15 0x00007f610965aa79 in ?? ()
#16 0x0000000000a8970d in __sanitizer::theDepot ()
#17 0x00007f61007233f9 in ?? ()
#18 0x00007f6103d2e6db in ?? ()
#19 0x00000fec9e80e688 in ?? ()
#20 0x00007f60f40b3460 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 9727):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x00007f60f60b8ca0 in ?? ()
#2 0x0000000000000051 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001e728 in ?? ()
#5 0x00007f60f60b8c90 in ?? ()
#6 0x00000000000000a2 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 9726):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 9725):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 9724):
#0 0x00007f6103d34fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f60f78bc450 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 9722):
#0 0x00007f6103d38d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20251211 20:56:34.772720 5967 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 2 with UUID 671b0c18e41544439fd978aa70f7ff53 and pid 10134
************************ BEGIN STACKS **************************
[New LWP 10136]
[New LWP 10137]
[New LWP 10138]
[New LWP 10139]
[New LWP 10145]
[New LWP 10146]
[New LWP 10147]
[New LWP 10150]
[New LWP 10151]
[New LWP 10152]
[New LWP 10153]
[New LWP 10154]
[New LWP 10155]
[New LWP 10156]
[New LWP 10157]
[New LWP 10158]
[New LWP 10159]
[New LWP 10160]
[New LWP 10161]
[New LWP 10162]
[New LWP 10163]
[New LWP 10164]
[New LWP 10165]
[New LWP 10166]
[New LWP 10167]
[New LWP 10168]
[New LWP 10169]
[New LWP 10170]
[New LWP 10171]
[New LWP 10172]
[New LWP 10173]
[New LWP 10174]
[New LWP 10175]
[New LWP 10176]
[New LWP 10177]
[New LWP 10178]
[New LWP 10179]
[New LWP 10180]
[New LWP 10181]
[New LWP 10182]
[New LWP 10183]
[New LWP 10184]
[New LWP 10185]
[New LWP 10186]
[New LWP 10187]
[New LWP 10188]
[New LWP 10189]
[New LWP 10190]
[New LWP 10191]
[New LWP 10192]
[New LWP 10193]
[New LWP 10194]
[New LWP 10195]
[New LWP 10196]
[New LWP 10197]
[New LWP 10198]
[New LWP 10199]
[New LWP 10200]
[New LWP 10201]
[New LWP 10202]
[New LWP 10203]
[New LWP 10204]
[New LWP 10205]
[New LWP 10206]
[New LWP 10207]
[New LWP 10208]
[New LWP 10209]
[New LWP 10210]
[New LWP 10211]
[New LWP 10212]
[New LWP 10213]
[New LWP 10214]
[New LWP 10215]
[New LWP 10216]
[New LWP 10217]
[New LWP 10218]
[New LWP 10219]
[New LWP 10220]
[New LWP 10221]
[New LWP 10222]
[New LWP 10223]
[New LWP 10224]
[New LWP 10225]
[New LWP 10226]
[New LWP 10227]
[New LWP 10228]
[New LWP 10229]
[New LWP 10230]
[New LWP 10231]
[New LWP 10232]
[New LWP 10233]
[New LWP 10234]
[New LWP 10235]
[New LWP 10236]
[New LWP 10237]
[New LWP 10238]
[New LWP 10239]
[New LWP 10240]
[New LWP 10241]
[New LWP 10242]
[New LWP 10243]
[New LWP 10244]
[New LWP 10245]
[New LWP 10246]
[New LWP 10247]
[New LWP 10248]
[New LWP 10249]
[New LWP 10250]
[New LWP 10251]
[New LWP 10252]
[New LWP 10253]
[New LWP 10254]
[New LWP 10255]
[New LWP 10256]
[New LWP 10257]
[New LWP 10258]
[New LWP 10259]
[New LWP 10260]
[New LWP 10261]
[New LWP 10262]
[New LWP 10265]
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c72
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c72
0x00007f61c47fed50 in ?? ()
Id Target Id Frame
* 1 LWP 10134 "kudu" 0x00007f61c47fed50 in ?? ()
2 LWP 10136 "kudu" 0x00007f61c47fafb9 in ?? ()
3 LWP 10137 "kudu" 0x00007f61c47fafb9 in ?? ()
4 LWP 10138 "kudu" 0x00007f61c47fafb9 in ?? ()
5 LWP 10139 "kernel-watcher-" 0x00007f61c47fafb9 in ?? ()
6 LWP 10145 "ntp client-1014" 0x00007f61c47fe9e2 in ?? ()
7 LWP 10146 "file cache-evic" 0x00007f61c47fafb9 in ?? ()
8 LWP 10147 "sq_acceptor" 0x00007f61bd318cb9 in ?? ()
9 LWP 10150 "rpc reactor-101" 0x00007f61bd325a47 in ?? ()
10 LWP 10151 "rpc reactor-101" 0x00007f61bd325a47 in ?? ()
11 LWP 10152 "rpc reactor-101" 0x00007f61bd325a47 in ?? ()
12 LWP 10153 "rpc reactor-101" 0x00007f61bd325a47 in ?? ()
13 LWP 10154 "MaintenanceMgr " 0x00007f61c47faad3 in ?? ()
14 LWP 10155 "txn-status-mana" 0x00007f61c47fafb9 in ?? ()
15 LWP 10156 "collect_and_rem" 0x00007f61c47fafb9 in ?? ()
16 LWP 10157 "tc-session-exp-" 0x00007f61c47fafb9 in ?? ()
17 LWP 10158 "rpc worker-1015" 0x00007f61c47faad3 in ?? ()
18 LWP 10159 "rpc worker-1015" 0x00007f61c47faad3 in ?? ()
19 LWP 10160 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
20 LWP 10161 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
21 LWP 10162 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
22 LWP 10163 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
23 LWP 10164 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
24 LWP 10165 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
25 LWP 10166 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
26 LWP 10167 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
27 LWP 10168 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
28 LWP 10169 "rpc worker-1016" 0x00007f61c47faad3 in ?? ()
29 LWP 10170 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
30 LWP 10171 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
31 LWP 10172 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
32 LWP 10173 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
33 LWP 10174 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
34 LWP 10175 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
35 LWP 10176 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
36 LWP 10177 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
37 LWP 10178 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
38 LWP 10179 "rpc worker-1017" 0x00007f61c47faad3 in ?? ()
39 LWP 10180 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
40 LWP 10181 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
41 LWP 10182 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
42 LWP 10183 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
43 LWP 10184 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
44 LWP 10185 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
45 LWP 10186 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
46 LWP 10187 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
47 LWP 10188 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
48 LWP 10189 "rpc worker-1018" 0x00007f61c47faad3 in ?? ()
49 LWP 10190 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
50 LWP 10191 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
51 LWP 10192 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
52 LWP 10193 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
53 LWP 10194 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
54 LWP 10195 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
55 LWP 10196 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
56 LWP 10197 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
57 LWP 10198 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
58 LWP 10199 "rpc worker-1019" 0x00007f61c47faad3 in ?? ()
59 LWP 10200 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
60 LWP 10201 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
61 LWP 10202 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
62 LWP 10203 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
63 LWP 10204 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
64 LWP 10205 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
65 LWP 10206 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
66 LWP 10207 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
67 LWP 10208 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
68 LWP 10209 "rpc worker-1020" 0x00007f61c47faad3 in ?? ()
69 LWP 10210 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
70 LWP 10211 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
71 LWP 10212 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
72 LWP 10213 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
73 LWP 10214 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
74 LWP 10215 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
75 LWP 10216 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
76 LWP 10217 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
77 LWP 10218 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
78 LWP 10219 "rpc worker-1021" 0x00007f61c47faad3 in ?? ()
79 LWP 10220 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
80 LWP 10221 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
81 LWP 10222 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
82 LWP 10223 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
83 LWP 10224 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
84 LWP 10225 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
85 LWP 10226 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
86 LWP 10227 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
87 LWP 10228 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
88 LWP 10229 "rpc worker-1022" 0x00007f61c47faad3 in ?? ()
89 LWP 10230 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
90 LWP 10231 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
91 LWP 10232 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
92 LWP 10233 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
93 LWP 10234 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
94 LWP 10235 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
95 LWP 10236 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
96 LWP 10237 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
97 LWP 10238 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
98 LWP 10239 "rpc worker-1023" 0x00007f61c47faad3 in ?? ()
99 LWP 10240 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
100 LWP 10241 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
101 LWP 10242 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
102 LWP 10243 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
103 LWP 10244 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
104 LWP 10245 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
105 LWP 10246 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
106 LWP 10247 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
107 LWP 10248 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
108 LWP 10249 "rpc worker-1024" 0x00007f61c47faad3 in ?? ()
109 LWP 10250 "rpc worker-1025" 0x00007f61c47faad3 in ?? ()
110 LWP 10251 "rpc worker-1025" 0x00007f61c47faad3 in ?? ()
111 LWP 10252 "rpc worker-1025" 0x00007f61c47faad3 in ?? ()
112 LWP 10253 "rpc worker-1025" 0x00007f61c47faad3 in ?? ()
113 LWP 10254 "rpc worker-1025" 0x00007f61c47faad3 in ?? ()
114 LWP 10255 "rpc worker-1025" 0x00007f61c47faad3 in ?? ()
115 LWP 10256 "rpc worker-1025" 0x00007f61c47faad3 in ?? ()
116 LWP 10257 "rpc worker-1025" 0x00007f61c47faad3 in ?? ()
117 LWP 10258 "diag-logger-102" 0x00007f61c47fafb9 in ?? ()
118 LWP 10259 "result-tracker-" 0x00007f61c47fafb9 in ?? ()
119 LWP 10260 "excess-log-dele" 0x00007f61c47fafb9 in ?? ()
120 LWP 10261 "acceptor-10261" 0x00007f61bd3270c7 in ?? ()
121 LWP 10262 "heartbeat-10262" 0x00007f61c47fafb9 in ?? ()
122 LWP 10265 "maintenance_sch" 0x00007f61c47fafb9 in ?? ()
Thread 122 (LWP 10265):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x00007f617967d700 in ?? ()
#2 0x0000000000000032 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000016ca0 in ?? ()
#5 0x00007f617967d750 in ?? ()
#6 0x0000000000000064 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 10262):
#0 0x00007f61c47fafb9 in ?? ()
#1 0xffe6cd9ab650d2a9 in ?? ()
#2 0x000000000000000c in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001a900 in ?? ()
#5 0x00007f617a6a4610 in ?? ()
#6 0x0000000000000018 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 10261):
#0 0x00007f61bd3270c7 in ?? ()
#1 0x00006140000216a8 in ?? ()
#2 0x00007f617aecbd70 in ?? ()
#3 0x00007f617aecbda0 in ?? ()
#4 0x00007f617aecbea0 in ?? ()
#5 0x00007f617aecbd90 in ?? ()
#6 0x00007f617aecbe00 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008c485b in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007f617aecbf20 in ?? ()
#11 0x00007f617aecb65c in ?? ()
#12 0x000000327aecb5d0 in ?? ()
#13 0x00007f617a6cf000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 10260):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x00007f617b6e5f60 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 10259):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x00007f617befe120 in ?? ()
#2 0x000000000000000c in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061100008abb0 in ?? ()
#5 0x00007f617befe110 in ?? ()
#6 0x0000000000000018 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 10258):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 10257):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000003 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007f617cf2deb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f617cf2ded0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007f61c47fa770 in ?? ()
#10 0x00007f617cf2ded0 in ?? ()
#11 0x00007f617cf2de90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 10256):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000005 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002a685c in ?? ()
#4 0x00007f617d745eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f617d745ed0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002a6810 in ?? ()
#9 0x00007f61c47fa770 in ?? ()
#10 0x00007f617d745ed0 in ?? ()
#11 0x00007f617d745e90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 10255):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 10254):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 10253):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 10252):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 10251):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 10250):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 10249):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 10248):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 10247):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 10246):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 10245):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 10244):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 10243):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 10242):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 10241):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 10240):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 10239):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 10238):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 10237):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 10236):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 10235):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 10234):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 10233):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 10232):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 10231):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 10230):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 10229):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 10228):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 10227):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 10226):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 10225):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 10224):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 10223):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 10222):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 10221):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 10220):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 10219):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 10218):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 10217):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007f61912eeeb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f61912eeed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 10216):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 74 (LWP 10215):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 10214):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 10213):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 10212):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 10211):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 10210):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 10209):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 10208):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 10207):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 10206):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 10205):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 10204):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 10203):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 10202):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 10201):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 10200):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 10199):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 10198):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 10197):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000120038 in ?? ()
#4 0x00007f619acb7eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f619acb7ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 10196):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 10195):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 10194):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 10193):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 10192):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 10191):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 10190):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 10189):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 10188):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 10187):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 10186):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 10185):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 10184):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 10183):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 10182):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 10181):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 10180):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 10179):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 10178):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 10177):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007f61a4e94eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f61a4e94ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 10176):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 34 (LWP 10175):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 10174):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 10173):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 10172):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 10171):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 10170):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 10169):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 10168):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 10167):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 10166):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 10165):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 10164):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 10163):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 10162):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 10161):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 10160):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 10159):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 10158):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 10157):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x00007f61aeebece0 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b1a0 in ?? ()
#5 0x00007f61aeebecd0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 10156):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x4008000000000000 in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001df18 in ?? ()
#5 0x00007f61af6c0270 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 14 (LWP 10155):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x00007f61afec0260 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000c5a8 in ?? ()
#5 0x00007f61afec0250 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 10154):
#0 0x00007f61c47faad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 10153):
#0 0x00007f61bd325a47 in ?? ()
#1 0x00007f61b0ec3340 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007f61b0ec3330 in ?? ()
#4 0x00007f61b0ec3540 in ?? ()
#5 0x00007f61b0ec3380 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007f61b0ec3400 in ?? ()
#8 0x00007f61bf99825d in ?? ()
#9 0x3fb962026d5fc000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x00000000ba000000 in ?? ()
#14 0x00007f6100000000 in ?? ()
#15 0x41da4ecba51ccc92 in ?? ()
#16 0x00000fecb61d0680 in ?? ()
#17 0x00007f61b0ec33e0 in ?? ()
#18 0x00007f61bf99cba3 in ?? ()
#19 0x00007f61b0ec33b0 in ?? ()
#20 0x3fb962026d5fc000 in ?? ()
#21 0x00000000b0ec3400 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb962026d5fc000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 10152):
#0 0x00007f61bd325a47 in ?? ()
#1 0x00007f61b16c4340 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007f61b16c4330 in ?? ()
#4 0x00007f61b16c4540 in ?? ()
#5 0x00007f61b16c4380 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007f61b16c4400 in ?? ()
#8 0x00007f61bf99825d in ?? ()
#9 0x3fb95d2726d29000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x000000000a000000 in ?? ()
#14 0x00007f6100000000 in ?? ()
#15 0x41da4ecba51ccc94 in ?? ()
#16 0x00000fecb62d0880 in ?? ()
#17 0x00007f61b16c43e0 in ?? ()
#18 0x00007f61bf99cba3 in ?? ()
#19 0x00007f61b16c43b0 in ?? ()
#20 0x3fb95d2726d29000 in ?? ()
#21 0x00000000b16c4400 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb95d2726d29000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 10151):
#0 0x00007f61bd325a47 in ?? ()
#1 0x00007f61b1ec5340 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007f61b1ec5330 in ?? ()
#4 0x00007f61b1ec5540 in ?? ()
#5 0x00007f61b1ec5380 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007f61b1ec5400 in ?? ()
#8 0x00007f61bf99825d in ?? ()
#9 0x3fb9854603258000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x00000000e4000000 in ?? ()
#14 0x00007f6100000000 in ?? ()
#15 0x41da4ecba51ccc95 in ?? ()
#16 0x00000fecb63d0a80 in ?? ()
#17 0x00007f61b1ec53e0 in ?? ()
#18 0x00007f61bf99cba3 in ?? ()
#19 0x00007f61b1ec53b0 in ?? ()
#20 0x3fb9854603258000 in ?? ()
#21 0x00000000b1ec5400 in ?? ()
#22 0x000061a00000ba80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb9854603258000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 10150):
#0 0x00007f61bd325a47 in ?? ()
#1 0x00007f61b3ab3340 in ?? ()
#2 0x000061a00000b480 in ?? ()
#3 0x00007f61b3ab3330 in ?? ()
#4 0x00007f61b3ab3540 in ?? ()
#5 0x00007f61b3ab3380 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007f61b3ab3400 in ?? ()
#8 0x00007f61bf99825d in ?? ()
#9 0x3fb9750ff326c000 in ?? ()
#10 0x000061a00000b480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000b480 in ?? ()
#13 0x00000000e7800000 in ?? ()
#14 0x00007f6100000000 in ?? ()
#15 0x41da4ecba51ccc95 in ?? ()
#16 0x00000fecb674e680 in ?? ()
#17 0x00007f61b3ab33e0 in ?? ()
#18 0x00007f61bf99cba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 10147):
#0 0x00007f61bd318cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f61b5ab77b8 in ?? ()
#3 0x000060200001ea50 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008bc4a1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 10146):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x00007f61b52b60e0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 10145):
#0 0x00007f61c47fe9e2 in ?? ()
#1 0x00007f61b42b2bc0 in ?? ()
#2 0x00007f61b42b2c60 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f61b42b2c10 in ?? ()
#6 0x00000000017ab3a0 in ?? ()
#7 0x00007f61b42b2c70 in ?? ()
#8 0x000061100008b510 in ?? ()
#9 0x00007f61b42b2c60 in ?? ()
#10 0x00000000008b6997 in __sanitizer::theDepot ()
#11 0x00007f61ca12861c in ?? ()
#12 0x00007f61ca117c29 in ?? ()
#13 0x00007f61ca11c216 in ?? ()
#14 0x00007f61ca120c50 in ?? ()
#15 0x00007f61ca120a79 in ?? ()
#16 0x0000000000a8970d in __sanitizer::theDepot ()
#17 0x00007f61c11e93f9 in ?? ()
#18 0x00007f61c47f46db in ?? ()
#19 0x00000fecb684e488 in ?? ()
#20 0x00007f61b42b2460 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 10139):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x00007f61b6ab8ca0 in ?? ()
#2 0x0000000000000043 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001e728 in ?? ()
#5 0x00007f61b6ab8c90 in ?? ()
#6 0x0000000000000086 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 10138):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 10137):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 10136):
#0 0x00007f61c47fafb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007f61b82bc450 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 10134):
#0 0x00007f61c47fed50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20251211 20:56:35.470979 5967 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 3 with UUID 6d21816e6f984a25a7eb64e5c8b29a2c and pid 9996
************************ BEGIN STACKS **************************
[New LWP 9999]
[New LWP 10000]
[New LWP 10001]
[New LWP 10002]
[New LWP 10008]
[New LWP 10009]
[New LWP 10010]
[New LWP 10013]
[New LWP 10014]
[New LWP 10015]
[New LWP 10016]
[New LWP 10017]
[New LWP 10018]
[New LWP 10020]
[New LWP 10021]
[New LWP 10022]
[New LWP 10023]
[New LWP 10024]
[New LWP 10025]
[New LWP 10026]
[New LWP 10027]
[New LWP 10028]
[New LWP 10029]
[New LWP 10030]
[New LWP 10031]
[New LWP 10032]
[New LWP 10033]
[New LWP 10034]
[New LWP 10035]
[New LWP 10036]
[New LWP 10037]
[New LWP 10038]
[New LWP 10039]
[New LWP 10040]
[New LWP 10041]
[New LWP 10042]
[New LWP 10043]
[New LWP 10044]
[New LWP 10045]
[New LWP 10046]
[New LWP 10047]
[New LWP 10048]
[New LWP 10049]
[New LWP 10050]
[New LWP 10051]
[New LWP 10052]
[New LWP 10053]
[New LWP 10054]
[New LWP 10055]
[New LWP 10056]
[New LWP 10057]
[New LWP 10058]
[New LWP 10059]
[New LWP 10060]
[New LWP 10061]
[New LWP 10062]
[New LWP 10063]
[New LWP 10064]
[New LWP 10065]
[New LWP 10066]
[New LWP 10067]
[New LWP 10068]
[New LWP 10069]
[New LWP 10070]
[New LWP 10071]
[New LWP 10072]
[New LWP 10073]
[New LWP 10074]
[New LWP 10075]
[New LWP 10076]
[New LWP 10077]
[New LWP 10078]
[New LWP 10079]
[New LWP 10080]
[New LWP 10081]
[New LWP 10082]
[New LWP 10083]
[New LWP 10084]
[New LWP 10085]
[New LWP 10086]
[New LWP 10087]
[New LWP 10088]
[New LWP 10089]
[New LWP 10090]
[New LWP 10091]
[New LWP 10092]
[New LWP 10093]
[New LWP 10094]
[New LWP 10095]
[New LWP 10096]
[New LWP 10097]
[New LWP 10098]
[New LWP 10099]
[New LWP 10100]
[New LWP 10101]
[New LWP 10102]
[New LWP 10103]
[New LWP 10104]
[New LWP 10105]
[New LWP 10106]
[New LWP 10107]
[New LWP 10108]
[New LWP 10109]
[New LWP 10110]
[New LWP 10111]
[New LWP 10112]
[New LWP 10113]
[New LWP 10114]
[New LWP 10115]
[New LWP 10116]
[New LWP 10117]
[New LWP 10118]
[New LWP 10119]
[New LWP 10120]
[New LWP 10121]
[New LWP 10122]
[New LWP 10123]
[New LWP 10124]
[New LWP 10125]
[New LWP 10126]
[New LWP 10131]
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c72
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c7a
Cannot access memory at address 0x2074736e6f633c72
0x00007fe9a14cfd50 in ?? ()
Id Target Id Frame
* 1 LWP 9996 "kudu" 0x00007fe9a14cfd50 in ?? ()
2 LWP 9999 "kudu" 0x00007fe9a14cbfb9 in ?? ()
3 LWP 10000 "kudu" 0x00007fe9a14cbfb9 in ?? ()
4 LWP 10001 "kudu" 0x00007fe9a14cbfb9 in ?? ()
5 LWP 10002 "kernel-watcher-" 0x00007fe9a14cbfb9 in ?? ()
6 LWP 10008 "ntp client-1000" 0x00007fe9a14cf9e2 in ?? ()
7 LWP 10009 "file cache-evic" 0x00007fe9a14cbfb9 in ?? ()
8 LWP 10010 "sq_acceptor" 0x00007fe999fe9cb9 in ?? ()
9 LWP 10013 "rpc reactor-100" 0x00007fe999ff6a47 in ?? ()
10 LWP 10014 "rpc reactor-100" 0x00007fe999ff6a47 in ?? ()
11 LWP 10015 "rpc reactor-100" 0x00007fe999ff6a47 in ?? ()
12 LWP 10016 "rpc reactor-100" 0x00007fe999ff6a47 in ?? ()
13 LWP 10017 "MaintenanceMgr " 0x00007fe9a14cbad3 in ?? ()
14 LWP 10018 "txn-status-mana" 0x00007fe9a14cbfb9 in ?? ()
15 LWP 10020 "collect_and_rem" 0x00007fe9a14cbfb9 in ?? ()
16 LWP 10021 "tc-session-exp-" 0x00007fe9a14cbfb9 in ?? ()
17 LWP 10022 "rpc worker-1002" 0x00007fe9a14cbad3 in ?? ()
18 LWP 10023 "rpc worker-1002" 0x00007fe9a14cbad3 in ?? ()
19 LWP 10024 "rpc worker-1002" 0x00007fe9a14cbad3 in ?? ()
20 LWP 10025 "rpc worker-1002" 0x00007fe9a14cbad3 in ?? ()
21 LWP 10026 "rpc worker-1002" 0x00007fe9a14cbad3 in ?? ()
22 LWP 10027 "rpc worker-1002" 0x00007fe9a14cbad3 in ?? ()
23 LWP 10028 "rpc worker-1002" 0x00007fe9a14cbad3 in ?? ()
24 LWP 10029 "rpc worker-1002" 0x00007fe9a14cbad3 in ?? ()
25 LWP 10030 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
26 LWP 10031 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
27 LWP 10032 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
28 LWP 10033 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
29 LWP 10034 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
30 LWP 10035 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
31 LWP 10036 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
32 LWP 10037 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
33 LWP 10038 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
34 LWP 10039 "rpc worker-1003" 0x00007fe9a14cbad3 in ?? ()
35 LWP 10040 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
36 LWP 10041 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
37 LWP 10042 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
38 LWP 10043 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
39 LWP 10044 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
40 LWP 10045 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
41 LWP 10046 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
42 LWP 10047 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
43 LWP 10048 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
44 LWP 10049 "rpc worker-1004" 0x00007fe9a14cbad3 in ?? ()
45 LWP 10050 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
46 LWP 10051 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
47 LWP 10052 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
48 LWP 10053 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
49 LWP 10054 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
50 LWP 10055 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
51 LWP 10056 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
52 LWP 10057 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
53 LWP 10058 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
54 LWP 10059 "rpc worker-1005" 0x00007fe9a14cbad3 in ?? ()
55 LWP 10060 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
56 LWP 10061 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
57 LWP 10062 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
58 LWP 10063 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
59 LWP 10064 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
60 LWP 10065 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
61 LWP 10066 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
62 LWP 10067 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
63 LWP 10068 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
64 LWP 10069 "rpc worker-1006" 0x00007fe9a14cbad3 in ?? ()
65 LWP 10070 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
66 LWP 10071 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
67 LWP 10072 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
68 LWP 10073 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
69 LWP 10074 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
70 LWP 10075 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
71 LWP 10076 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
72 LWP 10077 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
73 LWP 10078 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
74 LWP 10079 "rpc worker-1007" 0x00007fe9a14cbad3 in ?? ()
75 LWP 10080 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
76 LWP 10081 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
77 LWP 10082 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
78 LWP 10083 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
79 LWP 10084 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
80 LWP 10085 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
81 LWP 10086 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
82 LWP 10087 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
83 LWP 10088 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
84 LWP 10089 "rpc worker-1008" 0x00007fe9a14cbad3 in ?? ()
85 LWP 10090 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
86 LWP 10091 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
87 LWP 10092 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
88 LWP 10093 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
89 LWP 10094 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
90 LWP 10095 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
91 LWP 10096 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
92 LWP 10097 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
93 LWP 10098 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
94 LWP 10099 "rpc worker-1009" 0x00007fe9a14cbad3 in ?? ()
95 LWP 10100 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
96 LWP 10101 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
97 LWP 10102 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
98 LWP 10103 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
99 LWP 10104 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
100 LWP 10105 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
101 LWP 10106 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
102 LWP 10107 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
103 LWP 10108 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
104 LWP 10109 "rpc worker-1010" 0x00007fe9a14cbad3 in ?? ()
105 LWP 10110 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
106 LWP 10111 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
107 LWP 10112 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
108 LWP 10113 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
109 LWP 10114 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
110 LWP 10115 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
111 LWP 10116 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
112 LWP 10117 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
113 LWP 10118 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
114 LWP 10119 "rpc worker-1011" 0x00007fe9a14cbad3 in ?? ()
115 LWP 10120 "rpc worker-1012" 0x00007fe9a14cbad3 in ?? ()
116 LWP 10121 "rpc worker-1012" 0x00007fe9a14cbad3 in ?? ()
117 LWP 10122 "diag-logger-101" 0x00007fe9a14cbfb9 in ?? ()
118 LWP 10123 "result-tracker-" 0x00007fe9a14cbfb9 in ?? ()
119 LWP 10124 "excess-log-dele" 0x00007fe9a14cbfb9 in ?? ()
120 LWP 10125 "acceptor-10125" 0x00007fe999ff80c7 in ?? ()
121 LWP 10126 "heartbeat-10126" 0x00007fe9a14cbfb9 in ?? ()
122 LWP 10131 "maintenance_sch" 0x00007fe9a14cbfb9 in ?? ()
Thread 122 (LWP 10131):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x00007fe955aa3700 in ?? ()
#2 0x000000000000003b in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000016ca0 in ?? ()
#5 0x00007fe955aa3750 in ?? ()
#6 0x0000000000000076 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 10126):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0xffe6cd9ab650d2a9 in ?? ()
#2 0x0000000000000010 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001a900 in ?? ()
#5 0x00007fe956aca610 in ?? ()
#6 0x0000000000000020 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 10125):
#0 0x00007fe999ff80c7 in ?? ()
#1 0x00006140000214a8 in ?? ()
#2 0x00007fe9572e1d70 in ?? ()
#3 0x00007fe9572e1da0 in ?? ()
#4 0x00007fe9572e1ea0 in ?? ()
#5 0x00007fe9572e1d90 in ?? ()
#6 0x00007fe9572e1e00 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008c485b in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007fe9572e1f20 in ?? ()
#11 0x00007fe9572e165c in ?? ()
#12 0x00000032572e15d0 in ?? ()
#13 0x00007fe956ae5000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 10124):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x00007fe957b0af60 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 10123):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x00007fe958324120 in ?? ()
#2 0x000000000000000f in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061100008abb0 in ?? ()
#5 0x00007fe958324110 in ?? ()
#6 0x000000000000001e in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 10122):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 10121):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000003 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b684c in ?? ()
#4 0x00007fe959353eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe959353ed0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b6800 in ?? ()
#9 0x00007fe9a14cb770 in ?? ()
#10 0x00007fe959353ed0 in ?? ()
#11 0x00007fe959353e90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 115 (LWP 10120):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000005 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0002b004c in ?? ()
#4 0x00007fe959b6beb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe959b6bed0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0002b0000 in ?? ()
#9 0x00007fe9a14cb770 in ?? ()
#10 0x00007fe959b6bed0 in ?? ()
#11 0x00007fe959b6be90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 114 (LWP 10119):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 10118):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 10117):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 10116):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 10115):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 10114):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 10113):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 10112):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 10111):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 10110):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 10109):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 10108):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 10107):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 10106):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 10105):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 10104):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 10103):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 10102):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 10101):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 10100):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 10099):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 10098):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 10097):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 10096):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 10095):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 10094):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 10093):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 10092):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 10091):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 10090):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 10089):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 10088):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 10087):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 10086):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 10085):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 10084):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 10083):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 10082):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 10081):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x00000000000000fb in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007fe96cf4feb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe96cf4fed0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007fe9a14cb770 in ?? ()
#10 0x00007fe96cf4fed0 in ?? ()
#11 0x00007fe96cf4fe90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 10080):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000158 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007fe96d767eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe96d767ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 10079):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 73 (LWP 10078):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 10077):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 10076):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 10075):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 10074):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 10073):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 10072):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 10071):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 10070):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 10069):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 10068):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 10067):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 10066):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 10065):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 10064):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 10063):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 10062):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 10061):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007fe977130eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe977130ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 10060):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 54 (LWP 10059):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 53 (LWP 10058):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 52 (LWP 10057):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 10056):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 10055):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 10054):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 10053):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 10052):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 10051):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 10050):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 10049):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 10048):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 10047):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 10046):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 10045):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 10044):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 10043):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 10042):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 10041):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007fe981311eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe981311ed0 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007fe9a14cb770 in ?? ()
#10 0x00007fe981311ed0 in ?? ()
#11 0x00007fe981311e90 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 10040):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000046 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00009ffe8 in ?? ()
#4 0x00007fe981b29eb0 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fe981b29ed0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 34 (LWP 10039):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 33 (LWP 10038):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 10037):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 10036):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 10035):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 10034):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 10033):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 10032):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 10031):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 10030):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 10029):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 10028):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 10027):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 10026):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 10025):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 10024):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 10023):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 10022):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 10021):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x00007fe98b516ce0 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061300001b8a0 in ?? ()
#5 0x00007fe98b516cd0 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 15 (LWP 10020):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x0000000000003ff8 in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001df18 in ?? ()
#5 0x00007fe98bd5f270 in ?? ()
#6 0x0000000000000006 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 14 (LWP 10018):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x00007fe98cdaf260 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061800000c5a8 in ?? ()
#5 0x00007fe98cdaf250 in ?? ()
#6 0x0000000000000002 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 13 (LWP 10017):
#0 0x00007fe9a14cbad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 10016):
#0 0x00007fe999ff6a47 in ?? ()
#1 0x00007fe98ddb2340 in ?? ()
#2 0x000061a00000c680 in ?? ()
#3 0x00007fe98ddb2330 in ?? ()
#4 0x00007fe98ddb2540 in ?? ()
#5 0x00007fe98ddb2380 in ?? ()
#6 0x0000614000021698 in ?? ()
#7 0x00007fe98ddb2400 in ?? ()
#8 0x00007fe99c66925d in ?? ()
#9 0x3fb95cd307dbe000 in ?? ()
#10 0x000061a00000c680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c680 in ?? ()
#13 0x0000000074000000 in ?? ()
#14 0x00007fe900000000 in ?? ()
#15 0x41da4ecba51ccc94 in ?? ()
#16 0x00000ffdb1bae480 in ?? ()
#17 0x00007fe98ddb23e0 in ?? ()
#18 0x00007fe99c66dba3 in ?? ()
#19 0x00007fe98ddb23b0 in ?? ()
#20 0x3fb95cd307dbe000 in ?? ()
#21 0x000000008ddb2400 in ?? ()
#22 0x000061a00000c680 in ?? ()
#23 0x0000614000021698 in ?? ()
#24 0x3fb95cd307dbe000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 10015):
#0 0x00007fe999ff6a47 in ?? ()
#1 0x00007fe98e5b3340 in ?? ()
#2 0x000061a00000c080 in ?? ()
#3 0x00007fe98e5b3330 in ?? ()
#4 0x00007fe98e5b3540 in ?? ()
#5 0x00007fe98e5b3380 in ?? ()
#6 0x0000614000021498 in ?? ()
#7 0x00007fe98e5b3400 in ?? ()
#8 0x00007fe99c66925d in ?? ()
#9 0x3fb9831d5aa19000 in ?? ()
#10 0x000061a00000c080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000c080 in ?? ()
#13 0x000000009e000000 in ?? ()
#14 0x00007fe900000000 in ?? ()
#15 0x41da4ecba51ccc91 in ?? ()
#16 0x00000ffdb1cae680 in ?? ()
#17 0x00007fe98e5b33e0 in ?? ()
#18 0x00007fe99c66dba3 in ?? ()
#19 0x00007fe98e5b33b0 in ?? ()
#20 0x3fb9831d5aa19000 in ?? ()
#21 0x000000008e5b3400 in ?? ()
#22 0x000061a00000c080 in ?? ()
#23 0x0000614000021498 in ?? ()
#24 0x3fb9831d5aa19000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 10014):
#0 0x00007fe999ff6a47 in ?? ()
#1 0x00007fe98edb4340 in ?? ()
#2 0x000061a00000ba80 in ?? ()
#3 0x00007fe98edb4330 in ?? ()
#4 0x00007fe98edb4540 in ?? ()
#5 0x00007fe98edb4380 in ?? ()
#6 0x0000614000021298 in ?? ()
#7 0x00007fe98edb4400 in ?? ()
#8 0x00007fe99c66925d in ?? ()
#9 0x3fb9846aef774000 in ?? ()
#10 0x000061a00000ba80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ba80 in ?? ()
#13 0x00000000a9800000 in ?? ()
#14 0x00007fe900000000 in ?? ()
#15 0x41da4ecba51ccc96 in ?? ()
#16 0x00000ffdb1dae880 in ?? ()
#17 0x00007fe98edb43e0 in ?? ()
#18 0x00007fe99c66dba3 in ?? ()
#19 0x00007fe98edb43b0 in ?? ()
#20 0x3fb9846aef774000 in ?? ()
#21 0x000000008edb4400 in ?? ()
#22 0x000061a00000ba80 in ?? ()
#23 0x0000614000021298 in ?? ()
#24 0x3fb9846aef774000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 10013):
#0 0x00007fe999ff6a47 in ?? ()
#1 0x00007fe990dad340 in ?? ()
#2 0x000061a00000b480 in ?? ()
#3 0x00007fe990dad330 in ?? ()
#4 0x00007fe990dad540 in ?? ()
#5 0x00007fe990dad380 in ?? ()
#6 0x0000614000021098 in ?? ()
#7 0x00007fe990dad400 in ?? ()
#8 0x00007fe99c66925d in ?? ()
#9 0x3fb94f1596321000 in ?? ()
#10 0x000061a00000b480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000b480 in ?? ()
#13 0x000000001f000000 in ?? ()
#14 0x00007fe900000000 in ?? ()
#15 0x41da4ecba51ccc95 in ?? ()
#16 0x00000ffdb21ada80 in ?? ()
#17 0x00007fe990dad3e0 in ?? ()
#18 0x00007fe99c66dba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 10010):
#0 0x00007fe999fe9cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007fe9927b77b8 in ?? ()
#3 0x000060200001ea50 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008bc4a1 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 10009):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x00007fe991fb60e0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 10008):
#0 0x00007fe9a14cf9e2 in ?? ()
#1 0x00007fe9917b3bc0 in ?? ()
#2 0x00007fe9917b3c60 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007fe9917b3c10 in ?? ()
#6 0x00000000017ab3a0 in ?? ()
#7 0x00007fe9917b3c70 in ?? ()
#8 0x000061100008b510 in ?? ()
#9 0x00007fe9917b3c60 in ?? ()
#10 0x00000000008b6997 in __sanitizer::theDepot ()
#11 0x00007fe9a6df961c in ?? ()
#12 0x00007fe9a6de8c29 in ?? ()
#13 0x00007fe9a6ded216 in ?? ()
#14 0x00007fe9a6df1c50 in ?? ()
#15 0x00007fe9a6df1a79 in ?? ()
#16 0x0000000000a8970d in __sanitizer::theDepot ()
#17 0x00007fe99deba3f9 in ?? ()
#18 0x00007fe9a14c56db in ?? ()
#19 0x00000ffdb22ee688 in ?? ()
#20 0x00007fe9917b3460 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 10002):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x00007fe9937b8ca0 in ?? ()
#2 0x000000000000004e in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200001e728 in ?? ()
#5 0x00007fe9937b8c90 in ?? ()
#6 0x000000000000009c in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 10001):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 10000):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 9999):
#0 0x00007fe9a14cbfb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006120000126b8 in ?? ()
#5 0x00007fe994fbc450 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 9996):
#0 0x00007fe9a14cfd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20251211 20:56:36.180743 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 9859
I20251211 20:56:36.187132 9982 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:56:36.460510 10464 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5ccd26b0e8fe409f91fbd1da24bc7137)
I20251211 20:56:36.460805 10464 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:36.462451 10464 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 30c15ab665fb453fbea8cda8fd8e13d0 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 6d21816e6f984a25a7eb64e5c8b29a2c (127.5.211.196:35931), 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381)
I20251211 20:56:36.462797 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 9859
W20251211 20:56:36.508203 9740 connection.cc:537] client connection to 127.5.211.193:37733 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20251211 20:56:36.508504 9740 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:56:36.508769 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 9722
W20251211 20:56:36.509435 10466 negotiation.cc:337] Failed RPC negotiation. Trace:
1211 20:56:36.462648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.211.193:44861 (local address 127.5.211.196:35931)
1211 20:56:36.463735 (+ 1087us) server_negotiation.cc:207] Beginning negotiation
1211 20:56:36.463739 (+ 4us) server_negotiation.cc:400] Waiting for connection header
1211 20:56:36.508333 (+ 44594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.211.193:44861: BlockingRecv error: recv got EOF from 127.5.211.193:44861 (error 108)
Metrics: {"server-negotiator.queue_time_us":992,"thread_start_us":914,"threads_started":1}
W20251211 20:56:36.511298 9740 consensus_peers.cc:597] T fb7cb7e3685a4401a6d872d961a7fb17 P 5ccd26b0e8fe409f91fbd1da24bc7137 -> Peer 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733): Couldn't send request to peer 30c15ab665fb453fbea8cda8fd8e13d0. Status: Network error: Client connection negotiation failed: client connection to 127.5.211.193:37733: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20251211 20:56:36.514950 9846 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:56:36.780836 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 9722
I20251211 20:56:36.795365 10473 raft_consensus.cc:493] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5ccd26b0e8fe409f91fbd1da24bc7137)
I20251211 20:56:36.795612 10473 raft_consensus.cc:515] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d21816e6f984a25a7eb64e5c8b29a2c" member_type: VOTER last_known_addr { host: "127.5.211.196" port: 35931 } } peers { permanent_uuid: "5ccd26b0e8fe409f91fbd1da24bc7137" member_type: VOTER last_known_addr { host: "127.5.211.194" port: 45381 } } peers { permanent_uuid: "30c15ab665fb453fbea8cda8fd8e13d0" member_type: VOTER last_known_addr { host: "127.5.211.193" port: 37733 } }
I20251211 20:56:36.797201 10473 leader_election.cc:290] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381), 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733)
W20251211 20:56:36.798486 10015 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.211.193:37733: connect: Connection refused (error 111)
W20251211 20:56:36.799955 10015 leader_election.cc:336] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 30c15ab665fb453fbea8cda8fd8e13d0 (127.5.211.193:37733): Network error: Client connection negotiation failed: client connection to 127.5.211.193:37733: connect: Connection refused (error 111)
W20251211 20:56:36.827937 7216 connection.cc:537] server connection from 127.5.211.194:42503 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:56:36.828269 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 10134
W20251211 20:56:36.828583 10474 negotiation.cc:337] Failed RPC negotiation. Trace:
1211 20:56:36.797327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.211.194:45381 (local address 127.5.211.196:35531)
1211 20:56:36.797810 (+ 483us) negotiation.cc:107] Waiting for socket to connect
1211 20:56:36.797835 (+ 25us) client_negotiation.cc:174] Beginning negotiation
1211 20:56:36.798012 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
1211 20:56:36.828296 (+ 30284us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.211.194:45381: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":248,"threads_started":1}
W20251211 20:56:36.829751 10014 leader_election.cc:336] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 5ccd26b0e8fe409f91fbd1da24bc7137 (127.5.211.194:45381): Network error: Client connection negotiation failed: client connection to 127.5.211.194:45381: connect: Connection refused (error 111)
I20251211 20:56:36.830034 10014 leader_election.cc:304] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6d21816e6f984a25a7eb64e5c8b29a2c; no voters: 30c15ab665fb453fbea8cda8fd8e13d0, 5ccd26b0e8fe409f91fbd1da24bc7137
I20251211 20:56:36.830426 10473 raft_consensus.cc:2749] T fb7cb7e3685a4401a6d872d961a7fb17 P 6d21816e6f984a25a7eb64e5c8b29a2c [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20251211 20:56:36.834682 10256 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:56:36.922271 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 10134
W20251211 20:56:36.941121 7218 connection.cc:537] server connection from 127.5.211.195:60791 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20251211 20:56:36.941473 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 9996
I20251211 20:56:36.947535 10120 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:56:37.204517 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 9996
I20251211 20:56:37.249644 5967 external_mini_cluster.cc:1699] Attempting to check leaks for /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu pid 7200
I20251211 20:56:37.250758 7261 generic_service.cc:196] Checking for leaks (request via RPC)
I20251211 20:56:37.383313 5967 external_mini_cluster.cc:1658] Killing /tmp/dist-test-taskrKV6hb/build/asan/bin/kudu with pid 7200
2025-12-11T20:56:37Z chronyd exiting
I20251211 20:56:37.425673 5967 test_util.cc:183] -----------------------------------------------
I20251211 20:56:37.425784 5967 test_util.cc:184] Had failures, leaving test files at /tmp/dist-test-taskrKV6hb/test-tmp/maintenance_mode-itest.0.RollingRestartArgs_RollingRestartITest.TestWorkloads_4.1765486489393295-5967-0
[ FAILED ] RollingRestartArgs/RollingRestartITest.TestWorkloads/4, where GetParam() = 800-byte object <01-00 00-00 01-61 00-00 04-00 00-00 BE-BE BE-BE A0-24 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE C0-24 00-00 80-61 00-00 00-00 00-00 00-00 00-00 ... BE-BE BE-BE BE-BE BE-BE 78-27 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE 00-01 00-BE BE-BE BE-BE 04-00 00-00 03-00 00-00 01-BE BE-BE BE-BE BE-BE> (64215 ms)
[----------] 1 test from RollingRestartArgs/RollingRestartITest (64215 ms total)
[----------] Global test environment tear-down
[==========] 2 tests from 2 test suites ran. (107974 ms total)
[ PASSED ] 1 test.
[ FAILED ] 1 test, listed below:
[ FAILED ] RollingRestartArgs/RollingRestartITest.TestWorkloads/4, where GetParam() = 800-byte object <01-00 00-00 01-61 00-00 04-00 00-00 BE-BE BE-BE A0-24 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE C0-24 00-00 80-61 00-00 00-00 00-00 00-00 00-00 ... BE-BE BE-BE BE-BE BE-BE 78-27 00-00 80-61 00-00 00-00 00-00 00-00 00-00 00-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE BE-BE 00-01 00-BE BE-BE BE-BE 04-00 00-00 03-00 00-00 01-BE BE-BE BE-BE BE-BE>
1 FAILED TEST
I20251211 20:56:37.427294 5967 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/meta_cache.cc:302: suppressed but not reported on 12 messages since previous log ~17 seconds ago