[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from ReplaceTabletITest
[ RUN ] ReplaceTabletITest.ReplaceTabletsWhileWriting
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250114 08:56:21.776974 7514 test_util.cc:274] Using random seed: -1138019799
2025-01-14T08:56:21Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-14T08:56:21Z Disabled control of system clock
I20250114 08:56:21.808456 7514 external_mini_cluster.cc:1351] Running /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu
/tmp/dist-test-task4fHvvQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.86.190:36995
--webserver_interface=127.7.86.190
--webserver_port=0
--builtin_ntp_servers=127.7.86.148:37675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.86.190:36995 with env {}
W20250114 08:56:22.107843 7522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250114 08:56:22.108124 7522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250114 08:56:22.108258 7522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250114 08:56:22.115877 7522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250114 08:56:22.115988 7522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250114 08:56:22.116047 7522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250114 08:56:22.116128 7522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250114 08:56:22.125291 7522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.86.148:37675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.86.190:36995
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.86.190:36995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.86.190
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 622683201dc1b5c1b056d84565ee6cae8964ddd2
build type FASTDEBUG
built by None at 14 Jan 2025 08:43:25 UTC on 24a791456cd2
build id 4171
ASAN enabled
I20250114 08:56:22.126881 7522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250114 08:56:22.128552 7522 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250114 08:56:22.138350 7527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250114 08:56:22.139452 7528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250114 08:56:22.140000 7530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250114 08:56:22.140934 7522 server_base.cc:1034] running on GCE node
I20250114 08:56:22.141671 7522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250114 08:56:22.143024 7522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250114 08:56:22.144232 7522 hybrid_clock.cc:648] HybridClock initialized: now 1736844982144173 us; error 38 us; skew 500 ppm
I20250114 08:56:22.144546 7522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250114 08:56:22.146785 7522 webserver.cc:458] Webserver started at http://127.7.86.190:41709/ using document root <none> and password file <none>
I20250114 08:56:22.147505 7522 fs_manager.cc:362] Metadata directory not provided
I20250114 08:56:22.147624 7522 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250114 08:56:22.147897 7522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250114 08:56:22.150393 7522 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/data/instance:
uuid: "bf8c7ef6a2a547d79697f89825575301"
format_stamp: "Formatted at 2025-01-14 08:56:22 on dist-test-slave-kgk0"
I20250114 08:56:22.151149 7522 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/wal/instance:
uuid: "bf8c7ef6a2a547d79697f89825575301"
format_stamp: "Formatted at 2025-01-14 08:56:22 on dist-test-slave-kgk0"
I20250114 08:56:22.155956 7522 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250114 08:56:22.158982 7536 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:22.160189 7522 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20250114 08:56:22.160334 7522 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/data,/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/wal
uuid: "bf8c7ef6a2a547d79697f89825575301"
format_stamp: "Formatted at 2025-01-14 08:56:22 on dist-test-slave-kgk0"
I20250114 08:56:22.160504 7522 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250114 08:56:22.188346 7522 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250114 08:56:22.189172 7522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250114 08:56:22.189460 7522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250114 08:56:22.206745 7522 rpc_server.cc:307] RPC server started. Bound to: 127.7.86.190:36995
I20250114 08:56:22.206770 7587 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.86.190:36995 every 8 connection(s)
I20250114 08:56:22.208119 7522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/data/info.pb
I20250114 08:56:22.211678 7588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:22.212409 7514 external_mini_cluster.cc:1413] Started /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu as pid 7522
I20250114 08:56:22.212675 7514 external_mini_cluster.cc:1427] Reading /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/master-0/wal/instance
I20250114 08:56:22.221499 7588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301: Bootstrap starting.
I20250114 08:56:22.224304 7588 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:22.225235 7588 log.cc:826] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301: Log is configured to *not* fsync() on all Append() calls
I20250114 08:56:22.227761 7588 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301: No bootstrap required, opened a new log
I20250114 08:56:22.233901 7588 raft_consensus.cc:357] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8c7ef6a2a547d79697f89825575301" member_type: VOTER last_known_addr { host: "127.7.86.190" port: 36995 } }
I20250114 08:56:22.234247 7588 raft_consensus.cc:383] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:22.234344 7588 raft_consensus.cc:738] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8c7ef6a2a547d79697f89825575301, State: Initialized, Role: FOLLOWER
I20250114 08:56:22.235065 7588 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8c7ef6a2a547d79697f89825575301" member_type: VOTER last_known_addr { host: "127.7.86.190" port: 36995 } }
I20250114 08:56:22.235371 7588 raft_consensus.cc:397] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250114 08:56:22.235522 7588 raft_consensus.cc:491] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250114 08:56:22.235693 7588 raft_consensus.cc:3054] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:22.238333 7588 raft_consensus.cc:513] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8c7ef6a2a547d79697f89825575301" member_type: VOTER last_known_addr { host: "127.7.86.190" port: 36995 } }
I20250114 08:56:22.238804 7588 leader_election.cc:304] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf8c7ef6a2a547d79697f89825575301; no voters:
I20250114 08:56:22.239336 7588 leader_election.cc:290] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250114 08:56:22.239447 7593 raft_consensus.cc:2798] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [term 1 FOLLOWER]: Leader election won for term 1
I20250114 08:56:22.240053 7593 raft_consensus.cc:695] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [term 1 LEADER]: Becoming Leader. State: Replica: bf8c7ef6a2a547d79697f89825575301, State: Running, Role: LEADER
I20250114 08:56:22.240561 7593 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8c7ef6a2a547d79697f89825575301" member_type: VOTER last_known_addr { host: "127.7.86.190" port: 36995 } }
I20250114 08:56:22.241178 7588 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [sys.catalog]: configured and running, proceeding with master startup.
I20250114 08:56:22.243649 7594 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf8c7ef6a2a547d79697f89825575301" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8c7ef6a2a547d79697f89825575301" member_type: VOTER last_known_addr { host: "127.7.86.190" port: 36995 } } }
I20250114 08:56:22.243978 7594 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [sys.catalog]: This master's current role is: LEADER
I20250114 08:56:22.243633 7595 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf8c7ef6a2a547d79697f89825575301. Latest consensus state: current_term: 1 leader_uuid: "bf8c7ef6a2a547d79697f89825575301" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf8c7ef6a2a547d79697f89825575301" member_type: VOTER last_known_addr { host: "127.7.86.190" port: 36995 } } }
I20250114 08:56:22.244110 7595 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301 [sys.catalog]: This master's current role is: LEADER
I20250114 08:56:22.256385 7600 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250114 08:56:22.261523 7600 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250114 08:56:22.270294 7600 catalog_manager.cc:1348] Generated new cluster ID: 874302e2311d45bca2e8f65320b6804c
I20250114 08:56:22.270404 7600 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250114 08:56:22.278755 7600 catalog_manager.cc:1371] Generated new certificate authority record
I20250114 08:56:22.279857 7600 catalog_manager.cc:1505] Loading token signing keys...
I20250114 08:56:22.287782 7600 catalog_manager.cc:5899] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301: Generated new TSK 0
I20250114 08:56:22.288733 7600 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250114 08:56:22.300117 7514 external_mini_cluster.cc:1351] Running /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu
/tmp/dist-test-task4fHvvQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.86.129:0
--local_ip_for_outbound_sockets=127.7.86.129
--webserver_interface=127.7.86.129
--webserver_port=0
--tserver_master_addrs=127.7.86.190:36995
--builtin_ntp_servers=127.7.86.148:37675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20250114 08:56:22.606613 7612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250114 08:56:22.606882 7612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250114 08:56:22.607015 7612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250114 08:56:22.613945 7612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250114 08:56:22.614100 7612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.86.129
I20250114 08:56:22.623330 7612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.86.148:37675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.86.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.86.129
--webserver_port=0
--tserver_master_addrs=127.7.86.190:36995
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.86.129
--log_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 622683201dc1b5c1b056d84565ee6cae8964ddd2
build type FASTDEBUG
built by None at 14 Jan 2025 08:43:25 UTC on 24a791456cd2
build id 4171
ASAN enabled
I20250114 08:56:22.624815 7612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250114 08:56:22.626365 7612 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250114 08:56:22.638033 7618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250114 08:56:22.638993 7620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250114 08:56:22.639525 7617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250114 08:56:22.640235 7612 server_base.cc:1034] running on GCE node
I20250114 08:56:22.640784 7612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250114 08:56:22.641559 7612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250114 08:56:22.642766 7612 hybrid_clock.cc:648] HybridClock initialized: now 1736844982642700 us; error 39 us; skew 500 ppm
I20250114 08:56:22.643060 7612 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250114 08:56:22.649444 7612 webserver.cc:458] Webserver started at http://127.7.86.129:35097/ using document root <none> and password file <none>
I20250114 08:56:22.650103 7612 fs_manager.cc:362] Metadata directory not provided
I20250114 08:56:22.650215 7612 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250114 08:56:22.650465 7612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250114 08:56:22.652863 7612 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/data/instance:
uuid: "38213dd9fed845bf860fcb89a1fb8745"
format_stamp: "Formatted at 2025-01-14 08:56:22 on dist-test-slave-kgk0"
I20250114 08:56:22.653543 7612 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/wal/instance:
uuid: "38213dd9fed845bf860fcb89a1fb8745"
format_stamp: "Formatted at 2025-01-14 08:56:22 on dist-test-slave-kgk0"
I20250114 08:56:22.658301 7612 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20250114 08:56:22.661274 7626 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:22.662425 7612 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20250114 08:56:22.662607 7612 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/data,/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/wal
uuid: "38213dd9fed845bf860fcb89a1fb8745"
format_stamp: "Formatted at 2025-01-14 08:56:22 on dist-test-slave-kgk0"
I20250114 08:56:22.662781 7612 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250114 08:56:22.684566 7612 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250114 08:56:22.685356 7612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250114 08:56:22.685640 7612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250114 08:56:22.686539 7612 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250114 08:56:22.688035 7612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250114 08:56:22.688112 7612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:22.688241 7612 ts_tablet_manager.cc:610] Registered 0 tablets
I20250114 08:56:22.688295 7612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:22.719054 7612 rpc_server.cc:307] RPC server started. Bound to: 127.7.86.129:41225
I20250114 08:56:22.719099 7738 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.86.129:41225 every 8 connection(s)
I20250114 08:56:22.720336 7612 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/data/info.pb
I20250114 08:56:22.725419 7514 external_mini_cluster.cc:1413] Started /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu as pid 7612
I20250114 08:56:22.725623 7514 external_mini_cluster.cc:1427] Reading /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/wal/instance
I20250114 08:56:22.729077 7514 external_mini_cluster.cc:1351] Running /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu
/tmp/dist-test-task4fHvvQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.86.130:0
--local_ip_for_outbound_sockets=127.7.86.130
--webserver_interface=127.7.86.130
--webserver_port=0
--tserver_master_addrs=127.7.86.190:36995
--builtin_ntp_servers=127.7.86.148:37675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250114 08:56:22.733810 7739 heartbeater.cc:346] Connected to a master server at 127.7.86.190:36995
I20250114 08:56:22.734091 7739 heartbeater.cc:463] Registering TS with master...
I20250114 08:56:22.734953 7739 heartbeater.cc:510] Master 127.7.86.190:36995 requested a full tablet report, sending...
I20250114 08:56:22.737396 7553 ts_manager.cc:194] Registered new tserver with Master: 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225)
I20250114 08:56:22.739554 7553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.86.129:56641
W20250114 08:56:23.038362 7742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250114 08:56:23.038651 7742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250114 08:56:23.038797 7742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250114 08:56:23.045558 7742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250114 08:56:23.045763 7742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.86.130
I20250114 08:56:23.053741 7742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.86.148:37675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.86.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.86.130
--webserver_port=0
--tserver_master_addrs=127.7.86.190:36995
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.86.130
--log_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 622683201dc1b5c1b056d84565ee6cae8964ddd2
build type FASTDEBUG
built by None at 14 Jan 2025 08:43:25 UTC on 24a791456cd2
build id 4171
ASAN enabled
I20250114 08:56:23.055159 7742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250114 08:56:23.056775 7742 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250114 08:56:23.067126 7747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250114 08:56:23.067692 7748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250114 08:56:23.068581 7750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250114 08:56:23.069538 7742 server_base.cc:1034] running on GCE node
I20250114 08:56:23.070065 7742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250114 08:56:23.070887 7742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250114 08:56:23.072091 7742 hybrid_clock.cc:648] HybridClock initialized: now 1736844983072035 us; error 31 us; skew 500 ppm
I20250114 08:56:23.072467 7742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250114 08:56:23.074627 7742 webserver.cc:458] Webserver started at http://127.7.86.130:37995/ using document root <none> and password file <none>
I20250114 08:56:23.075280 7742 fs_manager.cc:362] Metadata directory not provided
I20250114 08:56:23.075438 7742 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250114 08:56:23.075721 7742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250114 08:56:23.078289 7742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/data/instance:
uuid: "96e3c2684cd4457685c687fcf04fce75"
format_stamp: "Formatted at 2025-01-14 08:56:23 on dist-test-slave-kgk0"
I20250114 08:56:23.079110 7742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/wal/instance:
uuid: "96e3c2684cd4457685c687fcf04fce75"
format_stamp: "Formatted at 2025-01-14 08:56:23 on dist-test-slave-kgk0"
I20250114 08:56:23.083952 7742 fs_manager.cc:696] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20250114 08:56:23.087082 7756 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:23.088232 7742 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.004s sys 0.000s
I20250114 08:56:23.088382 7742 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/data,/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/wal
uuid: "96e3c2684cd4457685c687fcf04fce75"
format_stamp: "Formatted at 2025-01-14 08:56:23 on dist-test-slave-kgk0"
I20250114 08:56:23.088546 7742 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250114 08:56:23.107296 7742 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250114 08:56:23.108035 7742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250114 08:56:23.108299 7742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250114 08:56:23.109081 7742 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250114 08:56:23.110388 7742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250114 08:56:23.110488 7742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:23.110626 7742 ts_tablet_manager.cc:610] Registered 0 tablets
I20250114 08:56:23.110680 7742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:23.138865 7742 rpc_server.cc:307] RPC server started. Bound to: 127.7.86.130:35427
I20250114 08:56:23.138917 7868 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.86.130:35427 every 8 connection(s)
I20250114 08:56:23.140033 7742 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/data/info.pb
I20250114 08:56:23.148384 7514 external_mini_cluster.cc:1413] Started /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu as pid 7742
I20250114 08:56:23.148566 7514 external_mini_cluster.cc:1427] Reading /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/wal/instance
I20250114 08:56:23.151322 7869 heartbeater.cc:346] Connected to a master server at 127.7.86.190:36995
I20250114 08:56:23.151566 7869 heartbeater.cc:463] Registering TS with master...
I20250114 08:56:23.151566 7514 external_mini_cluster.cc:1351] Running /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu
/tmp/dist-test-task4fHvvQ/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.86.131:0
--local_ip_for_outbound_sockets=127.7.86.131
--webserver_interface=127.7.86.131
--webserver_port=0
--tserver_master_addrs=127.7.86.190:36995
--builtin_ntp_servers=127.7.86.148:37675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250114 08:56:23.152284 7869 heartbeater.cc:510] Master 127.7.86.190:36995 requested a full tablet report, sending...
I20250114 08:56:23.153545 7553 ts_manager.cc:194] Registered new tserver with Master: 96e3c2684cd4457685c687fcf04fce75 (127.7.86.130:35427)
I20250114 08:56:23.154253 7553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.86.130:33511
W20250114 08:56:23.450556 7872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250114 08:56:23.450872 7872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250114 08:56:23.451013 7872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250114 08:56:23.458503 7872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250114 08:56:23.458809 7872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.86.131
I20250114 08:56:23.468102 7872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.86.148:37675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.86.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.86.131
--webserver_port=0
--tserver_master_addrs=127.7.86.190:36995
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.86.131
--log_dir=/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 622683201dc1b5c1b056d84565ee6cae8964ddd2
build type FASTDEBUG
built by None at 14 Jan 2025 08:43:25 UTC on 24a791456cd2
build id 4171
ASAN enabled
I20250114 08:56:23.469681 7872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250114 08:56:23.471412 7872 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250114 08:56:23.482606 7877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250114 08:56:23.483238 7880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250114 08:56:23.484086 7878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250114 08:56:23.484953 7872 server_base.cc:1034] running on GCE node
I20250114 08:56:23.485514 7872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250114 08:56:23.486341 7872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250114 08:56:23.487542 7872 hybrid_clock.cc:648] HybridClock initialized: now 1736844983487479 us; error 51 us; skew 500 ppm
I20250114 08:56:23.487815 7872 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250114 08:56:23.489882 7872 webserver.cc:458] Webserver started at http://127.7.86.131:41085/ using document root <none> and password file <none>
I20250114 08:56:23.490583 7872 fs_manager.cc:362] Metadata directory not provided
I20250114 08:56:23.490708 7872 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250114 08:56:23.490979 7872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250114 08:56:23.493458 7872 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/data/instance:
uuid: "15e6413d6f5047b8acc75dc385a3905e"
format_stamp: "Formatted at 2025-01-14 08:56:23 on dist-test-slave-kgk0"
I20250114 08:56:23.494220 7872 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/wal/instance:
uuid: "15e6413d6f5047b8acc75dc385a3905e"
format_stamp: "Formatted at 2025-01-14 08:56:23 on dist-test-slave-kgk0"
I20250114 08:56:23.499178 7872 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20250114 08:56:23.502398 7886 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:23.503721 7872 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20250114 08:56:23.503876 7872 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/data,/tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/wal
uuid: "15e6413d6f5047b8acc75dc385a3905e"
format_stamp: "Formatted at 2025-01-14 08:56:23 on dist-test-slave-kgk0"
I20250114 08:56:23.504053 7872 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250114 08:56:23.531816 7872 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250114 08:56:23.532640 7872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250114 08:56:23.532909 7872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250114 08:56:23.533834 7872 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250114 08:56:23.535256 7872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250114 08:56:23.535331 7872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:23.535458 7872 ts_tablet_manager.cc:610] Registered 0 tablets
I20250114 08:56:23.535511 7872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250114 08:56:23.567226 7872 rpc_server.cc:307] RPC server started. Bound to: 127.7.86.131:33401
I20250114 08:56:23.567303 7998 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.86.131:33401 every 8 connection(s)
I20250114 08:56:23.569597 7872 server_base.cc:1166] Dumped server information to /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/data/info.pb
I20250114 08:56:23.578100 7514 external_mini_cluster.cc:1413] Started /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu as pid 7872
I20250114 08:56:23.578275 7514 external_mini_cluster.cc:1427] Reading /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/wal/instance
I20250114 08:56:23.581869 7999 heartbeater.cc:346] Connected to a master server at 127.7.86.190:36995
I20250114 08:56:23.582185 7999 heartbeater.cc:463] Registering TS with master...
I20250114 08:56:23.582947 7999 heartbeater.cc:510] Master 127.7.86.190:36995 requested a full tablet report, sending...
I20250114 08:56:23.584270 7553 ts_manager.cc:194] Registered new tserver with Master: 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131:33401)
I20250114 08:56:23.585003 7553 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.86.131:50697
I20250114 08:56:23.594416 7514 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250114 08:56:23.621170 7514 test_util.cc:274] Using random seed: -1136175583
I20250114 08:56:23.640484 7553 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50844:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250114 08:56:23.642320 7553 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250114 08:56:23.672883 7803 tablet_service.cc:1467] Processing CreateTablet for tablet 8a95a443fc0649ffbc8bfedb25e9a299 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250114 08:56:23.672490 7804 tablet_service.cc:1467] Processing CreateTablet for tablet d3d3fe37b2484a118f4d5e6eb025f746 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250114 08:56:23.674525 7804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d3fe37b2484a118f4d5e6eb025f746. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.673833 7933 tablet_service.cc:1467] Processing CreateTablet for tablet 8a95a443fc0649ffbc8bfedb25e9a299 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250114 08:56:23.674587 7932 tablet_service.cc:1467] Processing CreateTablet for tablet 480d5a3e251c4d1c983303a97ed07a41 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250114 08:56:23.673903 7674 tablet_service.cc:1467] Processing CreateTablet for tablet d3d3fe37b2484a118f4d5e6eb025f746 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250114 08:56:23.675516 7674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d3fe37b2484a118f4d5e6eb025f746. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.675464 7931 tablet_service.cc:1467] Processing CreateTablet for tablet 3c6e55ceb8354ec9b789da6a49b5d0cd (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250114 08:56:23.675854 7803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a95a443fc0649ffbc8bfedb25e9a299. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.675865 7934 tablet_service.cc:1467] Processing CreateTablet for tablet d3d3fe37b2484a118f4d5e6eb025f746 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250114 08:56:23.676504 7932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 480d5a3e251c4d1c983303a97ed07a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.676610 7801 tablet_service.cc:1467] Processing CreateTablet for tablet 3c6e55ceb8354ec9b789da6a49b5d0cd (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250114 08:56:23.676715 7802 tablet_service.cc:1467] Processing CreateTablet for tablet 480d5a3e251c4d1c983303a97ed07a41 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250114 08:56:23.677059 7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c6e55ceb8354ec9b789da6a49b5d0cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.675801 7671 tablet_service.cc:1467] Processing CreateTablet for tablet 3c6e55ceb8354ec9b789da6a49b5d0cd (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250114 08:56:23.676142 7672 tablet_service.cc:1467] Processing CreateTablet for tablet 480d5a3e251c4d1c983303a97ed07a41 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250114 08:56:23.676499 7673 tablet_service.cc:1467] Processing CreateTablet for tablet 8a95a443fc0649ffbc8bfedb25e9a299 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250114 08:56:23.678120 7802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 480d5a3e251c4d1c983303a97ed07a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.678766 7934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3d3fe37b2484a118f4d5e6eb025f746. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.679545 7672 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 480d5a3e251c4d1c983303a97ed07a41. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.680054 7931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c6e55ceb8354ec9b789da6a49b5d0cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.680577 7933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a95a443fc0649ffbc8bfedb25e9a299. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.682832 7671 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c6e55ceb8354ec9b789da6a49b5d0cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.683617 7673 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a95a443fc0649ffbc8bfedb25e9a299. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:23.706491 8029 tablet_bootstrap.cc:492] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745: Bootstrap starting.
I20250114 08:56:23.711297 8029 tablet_bootstrap.cc:654] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.712059 8028 tablet_bootstrap.cc:492] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e: Bootstrap starting.
I20250114 08:56:23.712992 8029 log.cc:826] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745: Log is configured to *not* fsync() on all Append() calls
I20250114 08:56:23.717048 8028 tablet_bootstrap.cc:654] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.718640 8028 log.cc:826] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e: Log is configured to *not* fsync() on all Append() calls
I20250114 08:56:23.719925 8027 tablet_bootstrap.cc:492] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75: Bootstrap starting.
I20250114 08:56:23.722811 8027 tablet_bootstrap.cc:654] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.723846 8027 log.cc:826] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75: Log is configured to *not* fsync() on all Append() calls
I20250114 08:56:23.726011 8029 tablet_bootstrap.cc:492] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745: No bootstrap required, opened a new log
I20250114 08:56:23.726325 8029 ts_tablet_manager.cc:1397] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.007s
I20250114 08:56:23.727171 8027 tablet_bootstrap.cc:492] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75: No bootstrap required, opened a new log
I20250114 08:56:23.727494 8027 ts_tablet_manager.cc:1397] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20250114 08:56:23.728094 8028 tablet_bootstrap.cc:492] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e: No bootstrap required, opened a new log
I20250114 08:56:23.728344 8028 ts_tablet_manager.cc:1397] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.004s
I20250114 08:56:23.736352 8027 raft_consensus.cc:357] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.736804 8027 raft_consensus.cc:383] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.736697 8028 raft_consensus.cc:357] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.736941 8027 raft_consensus.cc:738] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e3c2684cd4457685c687fcf04fce75, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.737063 8028 raft_consensus.cc:383] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.737195 8028 raft_consensus.cc:738] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e6413d6f5047b8acc75dc385a3905e, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.737643 8027 consensus_queue.cc:260] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.737383 8029 raft_consensus.cc:357] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.737923 8029 raft_consensus.cc:383] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.737818 8028 consensus_queue.cc:260] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.738032 8029 raft_consensus.cc:738] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38213dd9fed845bf860fcb89a1fb8745, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.738935 7869 heartbeater.cc:502] Master 127.7.86.190:36995 was elected leader, sending a full tablet report...
I20250114 08:56:23.738821 8029 consensus_queue.cc:260] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.740185 7739 heartbeater.cc:502] Master 127.7.86.190:36995 was elected leader, sending a full tablet report...
I20250114 08:56:23.740610 8029 ts_tablet_manager.cc:1428] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20250114 08:56:23.741042 8029 tablet_bootstrap.cc:492] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: Bootstrap starting.
I20250114 08:56:23.743098 8029 tablet_bootstrap.cc:654] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.746591 8027 ts_tablet_manager.cc:1428] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75: Time spent starting tablet: real 0.019s user 0.007s sys 0.004s
I20250114 08:56:23.748662 8027 tablet_bootstrap.cc:492] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: Bootstrap starting.
I20250114 08:56:23.751140 8027 tablet_bootstrap.cc:654] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.751370 7999 heartbeater.cc:502] Master 127.7.86.190:36995 was elected leader, sending a full tablet report...
I20250114 08:56:23.751818 8028 ts_tablet_manager.cc:1428] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e: Time spent starting tablet: real 0.023s user 0.010s sys 0.014s
I20250114 08:56:23.752290 8028 tablet_bootstrap.cc:492] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e: Bootstrap starting.
I20250114 08:56:23.753105 8029 tablet_bootstrap.cc:492] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: No bootstrap required, opened a new log
I20250114 08:56:23.753274 8029 ts_tablet_manager.cc:1397] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.006s
I20250114 08:56:23.754184 8028 tablet_bootstrap.cc:654] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.754237 8029 raft_consensus.cc:357] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.754434 8029 raft_consensus.cc:383] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.754534 8029 raft_consensus.cc:738] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38213dd9fed845bf860fcb89a1fb8745, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.754850 8029 consensus_queue.cc:260] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.758023 8029 ts_tablet_manager.cc:1428] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20250114 08:56:23.759268 8029 tablet_bootstrap.cc:492] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745: Bootstrap starting.
I20250114 08:56:23.761487 8029 tablet_bootstrap.cc:654] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.762800 8027 tablet_bootstrap.cc:492] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: No bootstrap required, opened a new log
I20250114 08:56:23.762949 8027 ts_tablet_manager.cc:1397] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.006s
I20250114 08:56:23.763149 8028 tablet_bootstrap.cc:492] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e: No bootstrap required, opened a new log
I20250114 08:56:23.763285 8028 ts_tablet_manager.cc:1397] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20250114 08:56:23.763386 8029 tablet_bootstrap.cc:492] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745: No bootstrap required, opened a new log
I20250114 08:56:23.763564 8029 ts_tablet_manager.cc:1397] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20250114 08:56:23.763922 8027 raft_consensus.cc:357] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.764163 8027 raft_consensus.cc:383] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.764113 8029 raft_consensus.cc:357] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.764118 8028 raft_consensus.cc:357] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.764264 8027 raft_consensus.cc:738] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e3c2684cd4457685c687fcf04fce75, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.764336 8028 raft_consensus.cc:383] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.764355 8029 raft_consensus.cc:383] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.764434 8029 raft_consensus.cc:738] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38213dd9fed845bf860fcb89a1fb8745, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.764430 8028 raft_consensus.cc:738] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e6413d6f5047b8acc75dc385a3905e, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.764600 8027 consensus_queue.cc:260] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.764642 8028 consensus_queue.cc:260] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.764667 8029 consensus_queue.cc:260] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.765125 8027 ts_tablet_manager.cc:1428] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250114 08:56:23.765234 8028 ts_tablet_manager.cc:1428] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250114 08:56:23.765406 8027 tablet_bootstrap.cc:492] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75: Bootstrap starting.
I20250114 08:56:23.765458 8028 tablet_bootstrap.cc:492] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e: Bootstrap starting.
I20250114 08:56:23.765645 8029 ts_tablet_manager.cc:1428] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250114 08:56:23.765916 8029 tablet_bootstrap.cc:492] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745: Bootstrap starting.
I20250114 08:56:23.767148 8027 tablet_bootstrap.cc:654] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.767237 8028 tablet_bootstrap.cc:654] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.767596 8029 tablet_bootstrap.cc:654] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.781520 8028 tablet_bootstrap.cc:492] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e: No bootstrap required, opened a new log
I20250114 08:56:23.781699 8028 ts_tablet_manager.cc:1397] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.010s
I20250114 08:56:23.781919 8029 tablet_bootstrap.cc:492] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745: No bootstrap required, opened a new log
I20250114 08:56:23.782064 8029 ts_tablet_manager.cc:1397] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745: Time spent bootstrapping tablet: real 0.016s user 0.008s sys 0.008s
I20250114 08:56:23.782420 8027 tablet_bootstrap.cc:492] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75: No bootstrap required, opened a new log
I20250114 08:56:23.782624 8027 ts_tablet_manager.cc:1397] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.008s
I20250114 08:56:23.782522 8028 raft_consensus.cc:357] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.782780 8028 raft_consensus.cc:383] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.782742 8029 raft_consensus.cc:357] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.782867 8028 raft_consensus.cc:738] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e6413d6f5047b8acc75dc385a3905e, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.782941 8029 raft_consensus.cc:383] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.783035 8029 raft_consensus.cc:738] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38213dd9fed845bf860fcb89a1fb8745, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.783142 8028 consensus_queue.cc:260] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.783252 8029 consensus_queue.cc:260] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.783414 8027 raft_consensus.cc:357] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.783605 8027 raft_consensus.cc:383] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.783686 8027 raft_consensus.cc:738] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e3c2684cd4457685c687fcf04fce75, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.783715 8028 ts_tablet_manager.cc:1428] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250114 08:56:23.783931 8027 consensus_queue.cc:260] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.784104 8028 tablet_bootstrap.cc:492] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: Bootstrap starting.
I20250114 08:56:23.784603 8029 ts_tablet_manager.cc:1428] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250114 08:56:23.785823 8028 tablet_bootstrap.cc:654] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.786396 8027 ts_tablet_manager.cc:1428] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20250114 08:56:23.786783 8027 tablet_bootstrap.cc:492] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75: Bootstrap starting.
I20250114 08:56:23.789072 8027 tablet_bootstrap.cc:654] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:23.790032 8028 tablet_bootstrap.cc:492] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: No bootstrap required, opened a new log
I20250114 08:56:23.790166 8028 ts_tablet_manager.cc:1397] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20250114 08:56:23.790961 8027 tablet_bootstrap.cc:492] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75: No bootstrap required, opened a new log
I20250114 08:56:23.790915 8028 raft_consensus.cc:357] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.791106 8027 ts_tablet_manager.cc:1397] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s
I20250114 08:56:23.791115 8028 raft_consensus.cc:383] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.791201 8028 raft_consensus.cc:738] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e6413d6f5047b8acc75dc385a3905e, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.791466 8028 consensus_queue.cc:260] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.791862 8027 raft_consensus.cc:357] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.792013 8028 ts_tablet_manager.cc:1428] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250114 08:56:23.792074 8027 raft_consensus.cc:383] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:23.792171 8027 raft_consensus.cc:738] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e3c2684cd4457685c687fcf04fce75, State: Initialized, Role: FOLLOWER
I20250114 08:56:23.792366 8027 consensus_queue.cc:260] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.792871 8027 ts_tablet_manager.cc:1428] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
W20250114 08:56:23.821391 8000 tablet.cc:2367] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250114 08:56:23.821929 8033 raft_consensus.cc:491] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250114 08:56:23.822150 8033 raft_consensus.cc:513] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.823808 8033 leader_election.cc:290] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225), 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131:33401)
I20250114 08:56:23.832549 7694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a95a443fc0649ffbc8bfedb25e9a299" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745" is_pre_election: true
I20250114 08:56:23.832546 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a95a443fc0649ffbc8bfedb25e9a299" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15e6413d6f5047b8acc75dc385a3905e" is_pre_election: true
I20250114 08:56:23.832970 7694 raft_consensus.cc:2463] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 0.
I20250114 08:56:23.832969 7954 raft_consensus.cc:2463] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 0.
I20250114 08:56:23.833516 7760 leader_election.cc:304] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 38213dd9fed845bf860fcb89a1fb8745, 96e3c2684cd4457685c687fcf04fce75; no voters:
I20250114 08:56:23.833853 8033 raft_consensus.cc:2798] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250114 08:56:23.833988 8033 raft_consensus.cc:491] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250114 08:56:23.834084 8033 raft_consensus.cc:3054] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:23.836201 8033 raft_consensus.cc:513] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.836772 8033 leader_election.cc:290] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 election: Requested vote from peers 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225), 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131:33401)
I20250114 08:56:23.837163 7694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a95a443fc0649ffbc8bfedb25e9a299" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745"
I20250114 08:56:23.837222 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a95a443fc0649ffbc8bfedb25e9a299" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15e6413d6f5047b8acc75dc385a3905e"
I20250114 08:56:23.837320 7694 raft_consensus.cc:3054] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:23.837412 7954 raft_consensus.cc:3054] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:23.839416 7954 raft_consensus.cc:2463] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 1.
I20250114 08:56:23.839423 7694 raft_consensus.cc:2463] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 1.
I20250114 08:56:23.839773 7760 leader_election.cc:304] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 38213dd9fed845bf860fcb89a1fb8745, 96e3c2684cd4457685c687fcf04fce75; no voters:
I20250114 08:56:23.840076 8033 raft_consensus.cc:2798] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Leader election won for term 1
I20250114 08:56:23.840528 8033 raft_consensus.cc:695] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 1 LEADER]: Becoming Leader. State: Replica: 96e3c2684cd4457685c687fcf04fce75, State: Running, Role: LEADER
I20250114 08:56:23.840991 8033 consensus_queue.cc:237] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.845367 7553 catalog_manager.cc:5526] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96e3c2684cd4457685c687fcf04fce75 (127.7.86.130). New cstate: current_term: 1 leader_uuid: "96e3c2684cd4457685c687fcf04fce75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } health_report { overall_health: UNKNOWN } } }
I20250114 08:56:23.847095 8033 raft_consensus.cc:491] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250114 08:56:23.847245 8033 raft_consensus.cc:513] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.847949 8033 leader_election.cc:290] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225), 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131:33401)
I20250114 08:56:23.848177 7694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "480d5a3e251c4d1c983303a97ed07a41" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745" is_pre_election: true
I20250114 08:56:23.848244 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "480d5a3e251c4d1c983303a97ed07a41" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15e6413d6f5047b8acc75dc385a3905e" is_pre_election: true
I20250114 08:56:23.848414 7694 raft_consensus.cc:2463] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 0.
I20250114 08:56:23.848434 7954 raft_consensus.cc:2463] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 0.
I20250114 08:56:23.848760 7758 leader_election.cc:304] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 96e3c2684cd4457685c687fcf04fce75; no voters:
I20250114 08:56:23.849079 8033 raft_consensus.cc:2798] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250114 08:56:23.849172 8033 raft_consensus.cc:491] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250114 08:56:23.849277 8033 raft_consensus.cc:3054] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:23.851018 8033 raft_consensus.cc:513] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.851485 8033 leader_election.cc:290] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 election: Requested vote from peers 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225), 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131:33401)
I20250114 08:56:23.851827 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "480d5a3e251c4d1c983303a97ed07a41" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15e6413d6f5047b8acc75dc385a3905e"
I20250114 08:56:23.851837 7694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "480d5a3e251c4d1c983303a97ed07a41" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745"
I20250114 08:56:23.851992 7954 raft_consensus.cc:3054] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:23.852022 7694 raft_consensus.cc:3054] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:23.853668 7694 raft_consensus.cc:2463] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 1.
I20250114 08:56:23.853678 7954 raft_consensus.cc:2463] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 1.
I20250114 08:56:23.854014 7758 leader_election.cc:304] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 38213dd9fed845bf860fcb89a1fb8745, 96e3c2684cd4457685c687fcf04fce75; no voters:
I20250114 08:56:23.854278 8033 raft_consensus.cc:2798] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Leader election won for term 1
I20250114 08:56:23.854424 8033 raft_consensus.cc:695] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [term 1 LEADER]: Becoming Leader. State: Replica: 96e3c2684cd4457685c687fcf04fce75, State: Running, Role: LEADER
I20250114 08:56:23.854672 8033 consensus_queue.cc:237] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.857017 7553 catalog_manager.cc:5526] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96e3c2684cd4457685c687fcf04fce75 (127.7.86.130). New cstate: current_term: 1 leader_uuid: "96e3c2684cd4457685c687fcf04fce75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } health_report { overall_health: UNKNOWN } } }
W20250114 08:56:23.891919 7870 tablet.cc:2367] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250114 08:56:23.940013 8034 raft_consensus.cc:491] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250114 08:56:23.940194 8034 raft_consensus.cc:513] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.941798 8034 leader_election.cc:290] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225), 96e3c2684cd4457685c687fcf04fce75 (127.7.86.130:35427)
I20250114 08:56:23.947211 7824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d3fe37b2484a118f4d5e6eb025f746" candidate_uuid: "15e6413d6f5047b8acc75dc385a3905e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e3c2684cd4457685c687fcf04fce75" is_pre_election: true
I20250114 08:56:23.947641 7824 raft_consensus.cc:2463] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15e6413d6f5047b8acc75dc385a3905e in term 0.
I20250114 08:56:23.948207 7888 leader_election.cc:304] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 96e3c2684cd4457685c687fcf04fce75; no voters:
I20250114 08:56:23.948311 7694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d3fe37b2484a118f4d5e6eb025f746" candidate_uuid: "15e6413d6f5047b8acc75dc385a3905e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745" is_pre_election: true
I20250114 08:56:23.948518 7694 raft_consensus.cc:2463] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15e6413d6f5047b8acc75dc385a3905e in term 0.
I20250114 08:56:23.948573 8034 raft_consensus.cc:2798] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250114 08:56:23.948701 8034 raft_consensus.cc:491] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250114 08:56:23.948791 8034 raft_consensus.cc:3054] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:23.950667 8034 raft_consensus.cc:513] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.951145 8034 leader_election.cc:290] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [CANDIDATE]: Term 1 election: Requested vote from peers 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225), 96e3c2684cd4457685c687fcf04fce75 (127.7.86.130:35427)
I20250114 08:56:23.951515 7694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d3fe37b2484a118f4d5e6eb025f746" candidate_uuid: "15e6413d6f5047b8acc75dc385a3905e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745"
I20250114 08:56:23.951596 7824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d3fe37b2484a118f4d5e6eb025f746" candidate_uuid: "15e6413d6f5047b8acc75dc385a3905e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e3c2684cd4457685c687fcf04fce75"
I20250114 08:56:23.951683 7694 raft_consensus.cc:3054] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:23.951768 7824 raft_consensus.cc:3054] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:23.953361 7694 raft_consensus.cc:2463] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15e6413d6f5047b8acc75dc385a3905e in term 1.
I20250114 08:56:23.953552 7824 raft_consensus.cc:2463] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15e6413d6f5047b8acc75dc385a3905e in term 1.
I20250114 08:56:23.953682 7890 leader_election.cc:304] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 38213dd9fed845bf860fcb89a1fb8745; no voters:
I20250114 08:56:23.953938 8034 raft_consensus.cc:2798] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Leader election won for term 1
I20250114 08:56:23.954319 8034 raft_consensus.cc:695] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [term 1 LEADER]: Becoming Leader. State: Replica: 15e6413d6f5047b8acc75dc385a3905e, State: Running, Role: LEADER
I20250114 08:56:23.954746 8034 consensus_queue.cc:237] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:23.959045 7553 catalog_manager.cc:5526] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e reported cstate change: term changed from 0 to 1, leader changed from <none> to 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131). New cstate: current_term: 1 leader_uuid: "15e6413d6f5047b8acc75dc385a3905e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } health_report { overall_health: HEALTHY } } }
W20250114 08:56:23.972576 7740 tablet.cc:2367] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250114 08:56:24.049547 8034 raft_consensus.cc:491] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250114 08:56:24.049742 8034 raft_consensus.cc:513] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:24.050311 8034 leader_election.cc:290] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225), 96e3c2684cd4457685c687fcf04fce75 (127.7.86.130:35427)
I20250114 08:56:24.050845 7824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e55ceb8354ec9b789da6a49b5d0cd" candidate_uuid: "15e6413d6f5047b8acc75dc385a3905e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e3c2684cd4457685c687fcf04fce75" is_pre_election: true
I20250114 08:56:24.050850 7694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e55ceb8354ec9b789da6a49b5d0cd" candidate_uuid: "15e6413d6f5047b8acc75dc385a3905e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745" is_pre_election: true
I20250114 08:56:24.051116 7824 raft_consensus.cc:2463] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15e6413d6f5047b8acc75dc385a3905e in term 0.
I20250114 08:56:24.051117 7694 raft_consensus.cc:2463] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15e6413d6f5047b8acc75dc385a3905e in term 0.
I20250114 08:56:24.051487 7888 leader_election.cc:304] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 96e3c2684cd4457685c687fcf04fce75; no voters:
I20250114 08:56:24.051746 8034 raft_consensus.cc:2798] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250114 08:56:24.051838 8034 raft_consensus.cc:491] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250114 08:56:24.051921 8034 raft_consensus.cc:3054] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:24.053629 8034 raft_consensus.cc:513] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:24.054098 8034 leader_election.cc:290] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [CANDIDATE]: Term 1 election: Requested vote from peers 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225), 96e3c2684cd4457685c687fcf04fce75 (127.7.86.130:35427)
I20250114 08:56:24.054395 7694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e55ceb8354ec9b789da6a49b5d0cd" candidate_uuid: "15e6413d6f5047b8acc75dc385a3905e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745"
I20250114 08:56:24.054497 7824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c6e55ceb8354ec9b789da6a49b5d0cd" candidate_uuid: "15e6413d6f5047b8acc75dc385a3905e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e3c2684cd4457685c687fcf04fce75"
I20250114 08:56:24.054594 7694 raft_consensus.cc:3054] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:24.054728 7824 raft_consensus.cc:3054] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:24.056386 7694 raft_consensus.cc:2463] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15e6413d6f5047b8acc75dc385a3905e in term 1.
I20250114 08:56:24.056524 7824 raft_consensus.cc:2463] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15e6413d6f5047b8acc75dc385a3905e in term 1.
I20250114 08:56:24.056772 7890 leader_election.cc:304] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 38213dd9fed845bf860fcb89a1fb8745; no voters:
I20250114 08:56:24.057031 8034 raft_consensus.cc:2798] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Leader election won for term 1
I20250114 08:56:24.057186 8034 raft_consensus.cc:695] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [term 1 LEADER]: Becoming Leader. State: Replica: 15e6413d6f5047b8acc75dc385a3905e, State: Running, Role: LEADER
I20250114 08:56:24.057416 8034 consensus_queue.cc:237] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } }
I20250114 08:56:24.059839 7553 catalog_manager.cc:5526] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e reported cstate change: term changed from 0 to 1, leader changed from <none> to 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131). New cstate: current_term: 1 leader_uuid: "15e6413d6f5047b8acc75dc385a3905e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } health_report { overall_health: HEALTHY } } }
I20250114 08:56:24.172324 7694 raft_consensus.cc:1270] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Refusing update from remote peer 96e3c2684cd4457685c687fcf04fce75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250114 08:56:24.173696 7824 raft_consensus.cc:1270] T d3d3fe37b2484a118f4d5e6eb025f746 P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Refusing update from remote peer 15e6413d6f5047b8acc75dc385a3905e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250114 08:56:24.174361 7693 raft_consensus.cc:1270] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Refusing update from remote peer 15e6413d6f5047b8acc75dc385a3905e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250114 08:56:24.175140 7694 raft_consensus.cc:1270] T d3d3fe37b2484a118f4d5e6eb025f746 P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Refusing update from remote peer 15e6413d6f5047b8acc75dc385a3905e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250114 08:56:24.175056 8034 consensus_queue.cc:1035] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250114 08:56:24.175251 7954 raft_consensus.cc:1270] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Refusing update from remote peer 96e3c2684cd4457685c687fcf04fce75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250114 08:56:24.176317 8051 consensus_queue.cc:1035] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [LEADER]: Connected to new peer: Peer: permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250114 08:56:24.176529 8034 consensus_queue.cc:1035] T d3d3fe37b2484a118f4d5e6eb025f746 P 15e6413d6f5047b8acc75dc385a3905e [LEADER]: Connected to new peer: Peer: permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250114 08:56:24.176570 8033 consensus_queue.cc:1035] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250114 08:56:24.184830 7954 raft_consensus.cc:1270] T 480d5a3e251c4d1c983303a97ed07a41 P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Refusing update from remote peer 96e3c2684cd4457685c687fcf04fce75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250114 08:56:24.185449 8065 consensus_queue.cc:1035] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250114 08:56:24.186229 7824 raft_consensus.cc:1270] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Refusing update from remote peer 15e6413d6f5047b8acc75dc385a3905e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250114 08:56:24.187032 8034 consensus_queue.cc:1035] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250114 08:56:24.188181 8033 consensus_queue.cc:1035] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20250114 08:56:24.207412 7692 raft_consensus.cc:1270] T 480d5a3e251c4d1c983303a97ed07a41 P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Refusing update from remote peer 96e3c2684cd4457685c687fcf04fce75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250114 08:56:24.208120 8076 consensus_queue.cc:1035] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250114 08:56:24.212841 8075 mvcc.cc:204] Tried to move back new op lower bound from 7114117055158562816 to 7114117053816811520. Current Snapshot: MvccSnapshot[applied={T|T < 7114117055158562816}]
I20250114 08:56:24.214116 8061 mvcc.cc:204] Tried to move back new op lower bound from 7114117055171985408 to 7114117054701117440. Current Snapshot: MvccSnapshot[applied={T|T < 7114117055171985408}]
I20250114 08:56:24.237286 8067 mvcc.cc:204] Tried to move back new op lower bound from 7114117055158562816 to 7114117053816811520. Current Snapshot: MvccSnapshot[applied={T|T < 7114117055158562816}]
I20250114 08:56:26.857864 7548 master_service.cc:946] ReplaceTablet: received request to replace tablet 8a95a443fc0649ffbc8bfedb25e9a299 from {username='slave'} at 127.0.0.1:50842
I20250114 08:56:26.945122 7548 catalog_manager.cc:5813] T 00000000000000000000000000000000 P bf8c7ef6a2a547d79697f89825575301: Sending DeleteTablet for 3 replicas of tablet 8a95a443fc0649ffbc8bfedb25e9a299
I20250114 08:56:26.947136 7804 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a95a443fc0649ffbc8bfedb25e9a299 with delete_type TABLET_DATA_DELETED (replaced by tablet b9e44cce5bbe4d258ea9fa8ac8741fb4) from {username='slave'} at 127.0.0.1:46560
I20250114 08:56:26.947981 7548 catalog_manager.cc:6596] ReplaceTablet: tablet 8a95a443fc0649ffbc8bfedb25e9a299 deleted and replaced by tablet b9e44cce5bbe4d258ea9fa8ac8741fb4
I20250114 08:56:26.948158 7672 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a95a443fc0649ffbc8bfedb25e9a299 with delete_type TABLET_DATA_DELETED (replaced by tablet b9e44cce5bbe4d258ea9fa8ac8741fb4) from {username='slave'} at 127.0.0.1:43312
I20250114 08:56:26.949086 7932 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a95a443fc0649ffbc8bfedb25e9a299 with delete_type TABLET_DATA_DELETED (replaced by tablet b9e44cce5bbe4d258ea9fa8ac8741fb4) from {username='slave'} at 127.0.0.1:58380
I20250114 08:56:26.953305 7934 tablet_service.cc:1467] Processing CreateTablet for tablet b9e44cce5bbe4d258ea9fa8ac8741fb4 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250114 08:56:26.953783 7934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9e44cce5bbe4d258ea9fa8ac8741fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:26.958978 7674 tablet_service.cc:1467] Processing CreateTablet for tablet b9e44cce5bbe4d258ea9fa8ac8741fb4 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250114 08:56:26.959404 7674 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9e44cce5bbe4d258ea9fa8ac8741fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:26.959934 7802 tablet_service.cc:1467] Processing CreateTablet for tablet b9e44cce5bbe4d258ea9fa8ac8741fb4 (DEFAULT_TABLE table=test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250114 08:56:26.960451 7802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9e44cce5bbe4d258ea9fa8ac8741fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20250114 08:56:27.013566 8111 tablet_replica.cc:331] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: stopping tablet replica
I20250114 08:56:27.014073 8111 raft_consensus.cc:2238] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 1 LEADER]: Raft consensus shutting down.
I20250114 08:56:27.014765 8111 raft_consensus.cc:2267] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20250114 08:56:27.017481 8075 tablet_service.cc:695] failed op from {username='slave'} at 127.0.0.1:46570: Illegal state: RaftConsensus is not running: State = Stopped
I20250114 08:56:27.024150 8116 tablet_bootstrap.cc:492] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75: Bootstrap starting.
I20250114 08:56:27.027737 8116 tablet_bootstrap.cc:654] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:27.032284 8114 tablet_bootstrap.cc:492] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e: Bootstrap starting.
I20250114 08:56:27.034202 8114 tablet_bootstrap.cc:654] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:27.039606 8116 tablet_bootstrap.cc:492] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75: No bootstrap required, opened a new log
I20250114 08:56:27.040431 8116 ts_tablet_manager.cc:1397] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.004s
I20250114 08:56:27.041532 8116 raft_consensus.cc:357] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } }
I20250114 08:56:27.042944 8116 raft_consensus.cc:383] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:27.044951 8116 raft_consensus.cc:738] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e3c2684cd4457685c687fcf04fce75, State: Initialized, Role: FOLLOWER
I20250114 08:56:27.045410 8116 consensus_queue.cc:260] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } }
I20250114 08:56:27.048985 8116 ts_tablet_manager.cc:1428] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s
I20250114 08:56:27.053762 7548 master_service.cc:946] ReplaceTablet: received request to replace tablet 8a95a443fc0649ffbc8bfedb25e9a299 from {username='slave'} at 127.0.0.1:50842
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/replace_tablet-itest.cc:133: Failure
Failed
Bad status: Not found: Tablet 8a95a443fc0649ffbc8bfedb25e9a299 already deleted
I20250114 08:56:27.069129 8111 ts_tablet_manager.cc:1905] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250114 08:56:27.084736 8112 tablet_replica.cc:331] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: stopping tablet replica
I20250114 08:56:27.085314 8115 tablet_bootstrap.cc:492] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745: Bootstrap starting.
I20250114 08:56:27.086393 8112 raft_consensus.cc:2238] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Raft consensus shutting down.
I20250114 08:56:27.087020 8112 raft_consensus.cc:2267] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Raft consensus is shut down!
I20250114 08:56:27.090052 8111 ts_tablet_manager.cc:1918] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.219
I20250114 08:56:27.090221 8111 log.cc:1198] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: Deleting WAL directory at /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-1/wal/wals/8a95a443fc0649ffbc8bfedb25e9a299
I20250114 08:56:27.090840 8111 ts_tablet_manager.cc:1939] T 8a95a443fc0649ffbc8bfedb25e9a299 P 96e3c2684cd4457685c687fcf04fce75: Deleting consensus metadata
I20250114 08:56:27.093144 7538 catalog_manager.cc:4872] TS 96e3c2684cd4457685c687fcf04fce75 (127.7.86.130:35427): tablet 8a95a443fc0649ffbc8bfedb25e9a299 (table test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]) successfully deleted
I20250114 08:56:27.100375 8112 ts_tablet_manager.cc:1905] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250114 08:56:27.101301 8114 tablet_bootstrap.cc:492] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e: No bootstrap required, opened a new log
I20250114 08:56:27.101449 8114 ts_tablet_manager.cc:1397] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e: Time spent bootstrapping tablet: real 0.069s user 0.011s sys 0.026s
I20250114 08:56:27.102231 8114 raft_consensus.cc:357] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } }
I20250114 08:56:27.102418 8114 raft_consensus.cc:383] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:27.102497 8114 raft_consensus.cc:738] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15e6413d6f5047b8acc75dc385a3905e, State: Initialized, Role: FOLLOWER
I20250114 08:56:27.102705 8114 consensus_queue.cc:260] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } }
I20250114 08:56:27.103219 8114 ts_tablet_manager.cc:1428] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250114 08:56:27.103984 8115 tablet_bootstrap.cc:654] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745: Neither blocks nor log segments found. Creating new log.
I20250114 08:56:27.112376 8112 ts_tablet_manager.cc:1918] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.219
I20250114 08:56:27.112565 8112 log.cc:1198] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: Deleting WAL directory at /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-2/wal/wals/8a95a443fc0649ffbc8bfedb25e9a299
I20250114 08:56:27.112565 8113 tablet_replica.cc:331] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: stopping tablet replica
I20250114 08:56:27.113165 8112 ts_tablet_manager.cc:1939] T 8a95a443fc0649ffbc8bfedb25e9a299 P 15e6413d6f5047b8acc75dc385a3905e: Deleting consensus metadata
I20250114 08:56:27.115497 8113 raft_consensus.cc:2238] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250114 08:56:27.116384 7538 catalog_manager.cc:4872] TS 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131:33401): tablet 8a95a443fc0649ffbc8bfedb25e9a299 (table test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]) successfully deleted
I20250114 08:56:27.116626 8113 raft_consensus.cc:2267] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250114 08:56:27.119303 8113 ts_tablet_manager.cc:1905] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250114 08:56:27.121817 8115 tablet_bootstrap.cc:492] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745: No bootstrap required, opened a new log
I20250114 08:56:27.121973 8115 ts_tablet_manager.cc:1397] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745: Time spent bootstrapping tablet: real 0.037s user 0.007s sys 0.000s
I20250114 08:56:27.122800 8115 raft_consensus.cc:357] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } }
I20250114 08:56:27.123004 8115 raft_consensus.cc:383] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250114 08:56:27.123128 8115 raft_consensus.cc:738] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38213dd9fed845bf860fcb89a1fb8745, State: Initialized, Role: FOLLOWER
I20250114 08:56:27.123409 8115 consensus_queue.cc:260] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } }
I20250114 08:56:27.124387 8115 ts_tablet_manager.cc:1428] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20250114 08:56:27.125346 8113 ts_tablet_manager.cc:1918] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.216
I20250114 08:56:27.125516 8113 log.cc:1198] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: Deleting WAL directory at /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0/minicluster-data/ts-0/wal/wals/8a95a443fc0649ffbc8bfedb25e9a299
I20250114 08:56:27.126223 8113 ts_tablet_manager.cc:1939] T 8a95a443fc0649ffbc8bfedb25e9a299 P 38213dd9fed845bf860fcb89a1fb8745: Deleting consensus metadata
I20250114 08:56:27.127827 7540 catalog_manager.cc:4872] TS 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225): tablet 8a95a443fc0649ffbc8bfedb25e9a299 (table test-workload [id=2a2ace5f5bd64797b74e1b94e8c3b0bb]) successfully deleted
I20250114 08:56:27.222245 8105 raft_consensus.cc:491] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250114 08:56:27.222442 8105 raft_consensus.cc:513] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } }
I20250114 08:56:27.223124 8105 leader_election.cc:290] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131:33401), 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225)
I20250114 08:56:27.223569 7953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e44cce5bbe4d258ea9fa8ac8741fb4" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15e6413d6f5047b8acc75dc385a3905e" is_pre_election: true
I20250114 08:56:27.223712 7691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e44cce5bbe4d258ea9fa8ac8741fb4" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745" is_pre_election: true
I20250114 08:56:27.223811 7953 raft_consensus.cc:2463] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 0.
I20250114 08:56:27.223950 7691 raft_consensus.cc:2463] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 0.
I20250114 08:56:27.224256 7758 leader_election.cc:304] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 96e3c2684cd4457685c687fcf04fce75; no voters:
I20250114 08:56:27.224532 8105 raft_consensus.cc:2798] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250114 08:56:27.224623 8105 raft_consensus.cc:491] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250114 08:56:27.224710 8105 raft_consensus.cc:3054] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:27.227068 8105 raft_consensus.cc:513] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } }
I20250114 08:56:27.227628 8105 leader_election.cc:290] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 election: Requested vote from peers 15e6413d6f5047b8acc75dc385a3905e (127.7.86.131:33401), 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225)
I20250114 08:56:27.227986 7953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e44cce5bbe4d258ea9fa8ac8741fb4" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15e6413d6f5047b8acc75dc385a3905e"
I20250114 08:56:27.228063 7691 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9e44cce5bbe4d258ea9fa8ac8741fb4" candidate_uuid: "96e3c2684cd4457685c687fcf04fce75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38213dd9fed845bf860fcb89a1fb8745"
I20250114 08:56:27.228157 7953 raft_consensus.cc:3054] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:27.228243 7691 raft_consensus.cc:3054] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745 [term 0 FOLLOWER]: Advancing to term 1
I20250114 08:56:27.229956 7953 raft_consensus.cc:2463] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 15e6413d6f5047b8acc75dc385a3905e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 1.
I20250114 08:56:27.230130 7691 raft_consensus.cc:2463] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 38213dd9fed845bf860fcb89a1fb8745 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e3c2684cd4457685c687fcf04fce75 in term 1.
I20250114 08:56:27.230309 7758 leader_election.cc:304] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15e6413d6f5047b8acc75dc385a3905e, 96e3c2684cd4457685c687fcf04fce75; no voters:
I20250114 08:56:27.230628 8105 raft_consensus.cc:2798] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 1 FOLLOWER]: Leader election won for term 1
I20250114 08:56:27.230763 8105 raft_consensus.cc:695] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [term 1 LEADER]: Becoming Leader. State: Replica: 96e3c2684cd4457685c687fcf04fce75, State: Running, Role: LEADER
I20250114 08:56:27.231036 8105 consensus_queue.cc:237] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } }
I20250114 08:56:27.233611 7547 catalog_manager.cc:5526] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96e3c2684cd4457685c687fcf04fce75 (127.7.86.130). New cstate: current_term: 1 leader_uuid: "96e3c2684cd4457685c687fcf04fce75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "96e3c2684cd4457685c687fcf04fce75" member_type: VOTER last_known_addr { host: "127.7.86.130" port: 35427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 } health_report { overall_health: UNKNOWN } } }
I20250114 08:56:27.241951 8013 meta_cache.cc:557] tablet seems to be replaced: former ID 8a95a443fc0649ffbc8bfedb25e9a299, new ID b9e44cce5bbe4d258ea9fa8ac8741fb4 (table test-workload)
W20250114 08:56:27.242157 8013 batcher.cc:441] Invalid argument: Failed to write batch of 11 ops to tablet 8a95a443fc0649ffbc8bfedb25e9a299 after 3 attempt(s): tablet ID 8a95a443fc0649ffbc8bfedb25e9a299 is not valid
I20250114 08:56:27.242859 8013 meta_cache.cc:557] tablet seems to be replaced: former ID 8a95a443fc0649ffbc8bfedb25e9a299, new ID b9e44cce5bbe4d258ea9fa8ac8741fb4 (table test-workload)
I20250114 08:56:27.243687 8013 meta_cache.cc:557] tablet seems to be replaced: former ID 8a95a443fc0649ffbc8bfedb25e9a299, new ID b9e44cce5bbe4d258ea9fa8ac8741fb4 (table test-workload)
I20250114 08:56:27.250195 8013 meta_cache.cc:557] tablet seems to be replaced: former ID 8a95a443fc0649ffbc8bfedb25e9a299, new ID b9e44cce5bbe4d258ea9fa8ac8741fb4 (table test-workload)
I20250114 08:56:27.253017 7514 external_mini_cluster-itest-base.cc:80] Found fatal failure
I20250114 08:56:27.253188 7514 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 0 with UUID 38213dd9fed845bf860fcb89a1fb8745 and pid 7612
************************ BEGIN STACKS **************************
[New LWP 7613]
[New LWP 7614]
[New LWP 7615]
[New LWP 7616]
[New LWP 7622]
[New LWP 7623]
[New LWP 7624]
[New LWP 7627]
[New LWP 7628]
[New LWP 7629]
[New LWP 7630]
[New LWP 7631]
[New LWP 7632]
[New LWP 7633]
[New LWP 7634]
[New LWP 7635]
[New LWP 7636]
[New LWP 7637]
[New LWP 7638]
[New LWP 7639]
[New LWP 7640]
[New LWP 7641]
[New LWP 7642]
[New LWP 7643]
[New LWP 7644]
[New LWP 7645]
[New LWP 7646]
[New LWP 7647]
[New LWP 7648]
[New LWP 7649]
[New LWP 7650]
[New LWP 7651]
[New LWP 7652]
[New LWP 7653]
[New LWP 7654]
[New LWP 7655]
[New LWP 7656]
[New LWP 7657]
[New LWP 7658]
[New LWP 7659]
[New LWP 7660]
[New LWP 7661]
[New LWP 7662]
[New LWP 7663]
[New LWP 7664]
[New LWP 7665]
[New LWP 7666]
[New LWP 7667]
[New LWP 7668]
[New LWP 7669]
[New LWP 7670]
[New LWP 7671]
[New LWP 7672]
[New LWP 7673]
[New LWP 7674]
[New LWP 7675]
[New LWP 7676]
[New LWP 7677]
[New LWP 7678]
[New LWP 7679]
[New LWP 7680]
[New LWP 7681]
[New LWP 7682]
[New LWP 7683]
[New LWP 7684]
[New LWP 7685]
[New LWP 7686]
[New LWP 7687]
[New LWP 7688]
[New LWP 7689]
[New LWP 7690]
[New LWP 7691]
[New LWP 7692]
[New LWP 7693]
[New LWP 7694]
[New LWP 7695]
[New LWP 7696]
[New LWP 7697]
[New LWP 7698]
[New LWP 7699]
[New LWP 7700]
[New LWP 7701]
[New LWP 7702]
[New LWP 7703]
[New LWP 7704]
[New LWP 7705]
[New LWP 7706]
[New LWP 7707]
[New LWP 7708]
[New LWP 7709]
[New LWP 7710]
[New LWP 7711]
[New LWP 7712]
[New LWP 7713]
[New LWP 7714]
[New LWP 7715]
[New LWP 7716]
[New LWP 7717]
[New LWP 7718]
[New LWP 7719]
[New LWP 7720]
[New LWP 7721]
[New LWP 7722]
[New LWP 7723]
[New LWP 7724]
[New LWP 7725]
[New LWP 7726]
[New LWP 7727]
[New LWP 7728]
[New LWP 7729]
[New LWP 7730]
[New LWP 7731]
[New LWP 7732]
[New LWP 7733]
[New LWP 7734]
[New LWP 7735]
[New LWP 7736]
[New LWP 7737]
[New LWP 7738]
[New LWP 7739]
[New LWP 7740]
[New LWP 8069]
[New LWP 8071]
[New LWP 8079]
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
0x00007f82f908fd50 in ?? ()
Id Target Id Frame
* 1 LWP 7612 "kudu" 0x00007f82f908fd50 in ?? ()
2 LWP 7613 "kudu" 0x00007f82f908bfb9 in ?? ()
3 LWP 7614 "kudu" 0x00007f82f908bfb9 in ?? ()
4 LWP 7615 "kudu" 0x00007f82f908bfb9 in ?? ()
5 LWP 7616 "kernel-watcher-" 0x00007f82f908bfb9 in ?? ()
6 LWP 7622 "ntp client-7622" 0x00007f82f908f9e2 in ?? ()
7 LWP 7623 "file cache-evic" 0x00007f82f908bfb9 in ?? ()
8 LWP 7624 "sq_acceptor" 0x00007f82f1defcb9 in ?? ()
9 LWP 7627 "rpc reactor-762" 0x00007f82f1dfca47 in ?? ()
10 LWP 7628 "rpc reactor-762" 0x00007f82f1dfca47 in ?? ()
11 LWP 7629 "rpc reactor-762" 0x00007f82f1dfca47 in ?? ()
12 LWP 7630 "rpc reactor-763" 0x00007f82f1dfca47 in ?? ()
13 LWP 7631 "MaintenanceMgr " 0x00007f82f908bad3 in ?? ()
14 LWP 7632 "txn-status-mana" 0x00007f82f908bfb9 in ?? ()
15 LWP 7633 "collect_and_rem" 0x00007f82f908bfb9 in ?? ()
16 LWP 7634 "tc-session-exp-" 0x00007f82f908bfb9 in ?? ()
17 LWP 7635 "rpc worker-7635" 0x00007f82f908bad3 in ?? ()
18 LWP 7636 "rpc worker-7636" 0x00007f82f908bad3 in ?? ()
19 LWP 7637 "rpc worker-7637" 0x00007f82f908bad3 in ?? ()
20 LWP 7638 "rpc worker-7638" 0x00007f82f908bad3 in ?? ()
21 LWP 7639 "rpc worker-7639" 0x00007f82f908bad3 in ?? ()
22 LWP 7640 "rpc worker-7640" 0x00007f82f908bad3 in ?? ()
23 LWP 7641 "rpc worker-7641" 0x00007f82f908bad3 in ?? ()
24 LWP 7642 "rpc worker-7642" 0x00007f82f908bad3 in ?? ()
25 LWP 7643 "rpc worker-7643" 0x00007f82f908bad3 in ?? ()
26 LWP 7644 "rpc worker-7644" 0x00007f82f908bad3 in ?? ()
27 LWP 7645 "rpc worker-7645" 0x00007f82f908bad3 in ?? ()
28 LWP 7646 "rpc worker-7646" 0x00007f82f908bad3 in ?? ()
29 LWP 7647 "rpc worker-7647" 0x00007f82f908bad3 in ?? ()
30 LWP 7648 "rpc worker-7648" 0x00007f82f908bad3 in ?? ()
31 LWP 7649 "rpc worker-7649" 0x00007f82f908bad3 in ?? ()
32 LWP 7650 "rpc worker-7650" 0x00007f82f908bad3 in ?? ()
33 LWP 7651 "rpc worker-7651" 0x00007f82f908bad3 in ?? ()
34 LWP 7652 "rpc worker-7652" 0x00007f82f908bad3 in ?? ()
35 LWP 7653 "rpc worker-7653" 0x00007f82f908bad3 in ?? ()
36 LWP 7654 "rpc worker-7654" 0x00007f82f908bad3 in ?? ()
37 LWP 7655 "rpc worker-7655" 0x00007f82f908bad3 in ?? ()
38 LWP 7656 "rpc worker-7656" 0x00007f82f908bad3 in ?? ()
39 LWP 7657 "rpc worker-7657" 0x00007f82f908bad3 in ?? ()
40 LWP 7658 "rpc worker-7658" 0x00007f82f908bad3 in ?? ()
41 LWP 7659 "rpc worker-7659" 0x00007f82f908bad3 in ?? ()
42 LWP 7660 "rpc worker-7660" 0x00007f82f908bad3 in ?? ()
43 LWP 7661 "rpc worker-7661" 0x00007f82f908bad3 in ?? ()
44 LWP 7662 "rpc worker-7662" 0x00007f82f908bad3 in ?? ()
45 LWP 7663 "rpc worker-7663" 0x00007f82f908bad3 in ?? ()
46 LWP 7664 "rpc worker-7664" 0x00007f82f908bad3 in ?? ()
47 LWP 7665 "rpc worker-7665" 0x00007f82f908bad3 in ?? ()
48 LWP 7666 "rpc worker-7666" 0x00007f82f908bad3 in ?? ()
49 LWP 7667 "rpc worker-7667" 0x00007f82f908bad3 in ?? ()
50 LWP 7668 "rpc worker-7668" 0x00007f82f908bad3 in ?? ()
51 LWP 7669 "rpc worker-7669" 0x00007f82f908bad3 in ?? ()
52 LWP 7670 "rpc worker-7670" 0x00007f82f908bad3 in ?? ()
53 LWP 7671 "rpc worker-7671" 0x00007f82f908bad3 in ?? ()
54 LWP 7672 "rpc worker-7672" 0x00007f82f908bad3 in ?? ()
55 LWP 7673 "rpc worker-7673" 0x00007f82f908bad3 in ?? ()
56 LWP 7674 "rpc worker-7674" 0x00007f82f908bad3 in ?? ()
57 LWP 7675 "rpc worker-7675" 0x00007f82f908bad3 in ?? ()
58 LWP 7676 "rpc worker-7676" 0x00007f82f908bad3 in ?? ()
59 LWP 7677 "rpc worker-7677" 0x00007f82f908bad3 in ?? ()
60 LWP 7678 "rpc worker-7678" 0x00007f82f908bad3 in ?? ()
61 LWP 7679 "rpc worker-7679" 0x00007f82f908bad3 in ?? ()
62 LWP 7680 "rpc worker-7680" 0x00007f82f908bad3 in ?? ()
63 LWP 7681 "rpc worker-7681" 0x00007f82f908bad3 in ?? ()
64 LWP 7682 "rpc worker-7682" 0x00007f82f908bad3 in ?? ()
65 LWP 7683 "rpc worker-7683" 0x00007f82f908bad3 in ?? ()
66 LWP 7684 "rpc worker-7684" 0x00007f82f908bad3 in ?? ()
67 LWP 7685 "rpc worker-7685" 0x00007f82f908bad3 in ?? ()
68 LWP 7686 "rpc worker-7686" 0x00007f82f908bad3 in ?? ()
69 LWP 7687 "rpc worker-7687" 0x00007f82f908bad3 in ?? ()
70 LWP 7688 "rpc worker-7688" 0x00007f82f908bad3 in ?? ()
71 LWP 7689 "rpc worker-7689" 0x00007f82f908bad3 in ?? ()
72 LWP 7690 "rpc worker-7690" 0x00007f82f908bad3 in ?? ()
73 LWP 7691 "rpc worker-7691" 0x00007f82f908bad3 in ?? ()
74 LWP 7692 "rpc worker-7692" 0x00007f82f908bad3 in ?? ()
75 LWP 7693 "rpc worker-7693" 0x00007f82f908bad3 in ?? ()
76 LWP 7694 "rpc worker-7694" 0x00007f82f908bad3 in ?? ()
77 LWP 7695 "rpc worker-7695" 0x00007f82f908bad3 in ?? ()
78 LWP 7696 "rpc worker-7696" 0x00007f82f908bad3 in ?? ()
79 LWP 7697 "rpc worker-7697" 0x00007f82f908bad3 in ?? ()
80 LWP 7698 "rpc worker-7698" 0x00007f82f908bad3 in ?? ()
81 LWP 7699 "rpc worker-7699" 0x00007f82f908bad3 in ?? ()
82 LWP 7700 "rpc worker-7700" 0x00007f82f908bad3 in ?? ()
83 LWP 7701 "rpc worker-7701" 0x00007f82f908bad3 in ?? ()
84 LWP 7702 "rpc worker-7702" 0x00007f82f908bad3 in ?? ()
85 LWP 7703 "rpc worker-7703" 0x00007f82f908bad3 in ?? ()
86 LWP 7704 "rpc worker-7704" 0x00007f82f908bad3 in ?? ()
87 LWP 7705 "rpc worker-7705" 0x00007f82f908bad3 in ?? ()
88 LWP 7706 "rpc worker-7706" 0x00007f82f908bad3 in ?? ()
89 LWP 7707 "rpc worker-7707" 0x00007f82f908bad3 in ?? ()
90 LWP 7708 "rpc worker-7708" 0x00007f82f908bad3 in ?? ()
91 LWP 7709 "rpc worker-7709" 0x00007f82f908bad3 in ?? ()
92 LWP 7710 "rpc worker-7710" 0x00007f82f908bad3 in ?? ()
93 LWP 7711 "rpc worker-7711" 0x00007f82f908bad3 in ?? ()
94 LWP 7712 "rpc worker-7712" 0x00007f82f908bad3 in ?? ()
95 LWP 7713 "rpc worker-7713" 0x00007f82f908bad3 in ?? ()
96 LWP 7714 "rpc worker-7714" 0x00007f82f908bad3 in ?? ()
97 LWP 7715 "rpc worker-7715" 0x00007f82f908bad3 in ?? ()
98 LWP 7716 "rpc worker-7716" 0x00007f82f908bad3 in ?? ()
99 LWP 7717 "rpc worker-7717" 0x00007f82f908bad3 in ?? ()
100 LWP 7718 "rpc worker-7718" 0x00007f82f908bad3 in ?? ()
101 LWP 7719 "rpc worker-7719" 0x00007f82f908bad3 in ?? ()
102 LWP 7720 "rpc worker-7720" 0x00007f82f908bad3 in ?? ()
103 LWP 7721 "rpc worker-7721" 0x00007f82f908bad3 in ?? ()
104 LWP 7722 "rpc worker-7722" 0x00007f82f908bad3 in ?? ()
105 LWP 7723 "rpc worker-7723" 0x00007f82f908bad3 in ?? ()
106 LWP 7724 "rpc worker-7724" 0x00007f82f908bad3 in ?? ()
107 LWP 7725 "rpc worker-7725" 0x00007f82f908bad3 in ?? ()
108 LWP 7726 "rpc worker-7726" 0x00007f82f908bad3 in ?? ()
109 LWP 7727 "rpc worker-7727" 0x00007f82f908bad3 in ?? ()
110 LWP 7728 "rpc worker-7728" 0x00007f82f908bad3 in ?? ()
111 LWP 7729 "rpc worker-7729" 0x00007f82f908bad3 in ?? ()
112 LWP 7730 "rpc worker-7730" 0x00007f82f908bad3 in ?? ()
113 LWP 7731 "rpc worker-7731" 0x00007f82f908bad3 in ?? ()
114 LWP 7732 "rpc worker-7732" 0x00007f82f908bad3 in ?? ()
115 LWP 7733 "rpc worker-7733" 0x00007f82f908bad3 in ?? ()
116 LWP 7734 "rpc worker-7734" 0x00007f82f908bad3 in ?? ()
117 LWP 7735 "diag-logger-773" 0x00007f82f5b20efe in ?? ()
118 LWP 7736 "result-tracker-" 0x00007f82f908bfb9 in ?? ()
119 LWP 7737 "excess-log-dele" 0x00007f82f908bfb9 in ?? ()
120 LWP 7738 "acceptor-7738" 0x00007f82f1dfe0c7 in ?? ()
121 LWP 7739 "heartbeat-7739" 0x00007f82f908bfb9 in ?? ()
122 LWP 7740 "maintenance_sch" 0x00007f82f908bfb9 in ?? ()
123 LWP 8069 "wal-append [wor" 0x00007f82f908bfb9 in ?? ()
124 LWP 8071 "wal-append [wor" 0x00007f82f908bfb9 in ?? ()
125 LWP 8079 "wal-append [wor" 0x00007f82f908bfb9 in ?? ()
Thread 125 (LWP 8079):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x000060200015a318 in ?? ()
#2 0x0000000000000134 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061a000046fb0 in ?? ()
#5 0x00007f82a773b090 in ?? ()
#6 0x0000000000000268 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 124 (LWP 8071):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x000060200014f6c0 in ?? ()
#2 0x000000000000014f in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x000061a0000463b4 in ?? ()
#5 0x00007f82a7f53090 in ?? ()
#6 0x000000000000029f in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f82a7f530b0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 123 (LWP 8069):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x0000602000152658 in ?? ()
#2 0x0000000000000149 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x000061a0000421b4 in ?? ()
#5 0x00007f82a8f93090 in ?? ()
#6 0x0000000000000293 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f82a8f930b0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 122 (LWP 7740):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f82ae14e550 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 7739):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x9fa35ef76db3e6dd in ?? ()
#2 0x0000000000000009 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007f82ae9665b0 in ?? ()
#6 0x0000000000000013 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f82ae9665d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f82ae966650 in ?? ()
#12 0x00000ff055d2cccc in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 7738):
#0 0x00007f82f1dfe0c7 in ?? ()
#1 0x00006140000224a8 in ?? ()
#2 0x00007f82af18dd50 in ?? ()
#3 0x00007f82af18dd80 in ?? ()
#4 0x00007f82af18de80 in ?? ()
#5 0x00007f82af18dd70 in ?? ()
#6 0x00007f82af18dde0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007f82af18df00 in ?? ()
#11 0x00007f82af18d63c in ?? ()
#12 0x00000032af18d5b0 in ?? ()
#13 0x00007f82ae991000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 7737):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x00007f82af9a7f00 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 7736):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x00007f82b01c00c0 in ?? ()
#2 0x0000000000000005 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000879b0 in ?? ()
#5 0x00007f82b01c00b0 in ?? ()
#6 0x000000000000000a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 7735):
#0 0x00007f82f5b20efe in ?? ()
#1 0x00000ff05613ad89 in ?? ()
#2 0x00007f82b09d6c80 in ?? ()
#3 0x0000608000317ca0 in ?? ()
#4 0x00007f82b09d6c68 in ?? ()
#5 0x00007f82b09d6c40 in ?? ()
#6 0x00007f82b09d6c90 in ?? ()
#7 0x00007f82b09d6c4c in ?? ()
#8 0x00007f82b09d6e40 in ?? ()
#9 0x000000000177bae0 in ?? ()
#10 0x00007f82b09d6c40 in ?? ()
#11 0x9ddfea08eb382d69 in ?? ()
#12 0x00006060000f42b0 in ?? ()
#13 0x00007f82b09d6ef0 in ?? ()
#14 0x00007f82f5bab1c3 in ?? ()
#15 0x0000000041b58ab3 in ?? ()
#16 0x00007f82f55fcd7b in ?? ()
#17 0x00007f82f5baa800 in ?? ()
#18 0x00007f82f5b3c20d in ?? ()
#19 0x0000000003938700 in ?? ()
#20 0x00000a0000000002 in ?? ()
#21 0x0000010000000013 in ?? ()
#22 0x0000008000904a07 in ?? ()
#23 0x00006030000000ff in ?? ()
#24 0x0000000000000000 in ?? ()
Thread 116 (LWP 7734):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 115 (LWP 7733):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 114 (LWP 7732):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 7731):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 7730):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 7729):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 7728):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 7727):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 7726):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 7725):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 7724):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 7723):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 7722):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 7721):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 7720):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 7719):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 7718):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 7717):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 7716):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 7715):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 7714):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 7713):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 7712):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 7711):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 7710):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 7709):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 7708):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 7707):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 7706):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 7705):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 7704):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 7703):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 7702):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 7701):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 7700):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 7699):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 7698):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 7697):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 7696):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 7695):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 7694):
#0 0x00007f82f908bad3 in ?? ()
#1 0x00000000000000c9 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007f82c55b0e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82c55b0e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007f82f908b770 in ?? ()
#10 0x00007f82c55b0e70 in ?? ()
#11 0x00000ff058ab61d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 7693):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000073 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f82c5dc8e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82c5dc8e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f82f908b770 in ?? ()
#10 0x00007f82c5dc8e70 in ?? ()
#11 0x00000ff058bb91d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 74 (LWP 7692):
#0 0x00007f82f908bad3 in ?? ()
#1 0x00000000000000b4 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a0088 in ?? ()
#4 0x00007f82c65e0e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82c65e0e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 73 (LWP 7691):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000090 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001967c8 in ?? ()
#4 0x00007f82c6df8e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82c6df8e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 72 (LWP 7690):
#0 0x00007f82f908bad3 in ?? ()
#1 0x00000000000000b4 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00018ffc8 in ?? ()
#4 0x00007f82c7610e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82c7610e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 71 (LWP 7689):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 7688):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 7687):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 7686):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 7685):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 7684):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 7683):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 7682):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 7681):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 7680):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 7679):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 7678):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 7677):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 7676):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 7675):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 7674):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007f82cf791e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82cf791e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 7673):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f82cffa9e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82cffa9e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f82f908b770 in ?? ()
#10 0x00007f82cffa9e70 in ?? ()
#11 0x00000ff059ff53d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 54 (LWP 7672):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00011d048 in ?? ()
#4 0x00007f82d07c1e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82d07c1e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 53 (LWP 7671):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00011684c in ?? ()
#4 0x00007f82d0fd9e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82d0fd9e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d000116800 in ?? ()
#9 0x00007f82f908b770 in ?? ()
#10 0x00007f82d0fd9e70 in ?? ()
#11 0x00000ff05a1fb3d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 52 (LWP 7670):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 7669):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 7668):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 7667):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 7666):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 7665):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 7664):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 7663):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 7662):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 7661):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 7660):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 7659):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 7658):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 7657):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 7656):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 7655):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 7654):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000003 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f82d9972e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82d9972e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f82f908b770 in ?? ()
#10 0x00007f82d9972e70 in ?? ()
#11 0x00000ff05b32e5d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 7653):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f82da18ae50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82da18ae70 in ?? ()
I20250114 08:56:27.790617 8105 consensus_queue.cc:1035] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15e6413d6f5047b8acc75dc385a3905e" member_type: VOTER last_known_addr { host: "127.7.86.131" port: 33401 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f82f908b770 in ?? ()
#10 0x00007f82da18ae70 in ?? ()
#11 0x00000ff05b4315d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 34 (LWP 7652):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000967fc in ?? ()
#4 0x00007f82da9a2e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f82da9a2e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000967b0 in ?? ()
#9 0x00007f82f908b770 in ?? ()
#10 0x00007f82da9a2e70 in ?? ()
#11 0x00000ff05b5345d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 7651):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 32 (LWP 7650):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 7649):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 7648):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 7647):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 7646):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 7645):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 7644):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 7643):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 7642):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 7641):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 7640):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 7639):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 7638):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 7637):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 7636):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 7635):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 7634):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x00007f82e3b87c80 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 15 (LWP 7633):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 7632):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x00007f82e4baf1c0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 13 (LWP 7631):
#0 0x00007f82f908bad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 7630):
#0 0x00007f82f1dfca47 in ?? ()
#1 0x00007f82e5bb22e0 in ?? ()
#2 0x000061a00000f680 in ?? ()
#3 0x00007f82e5bb22d0 in ?? ()
#4 0x00007f82e5bb24e0 in ?? ()
#5 0x00007f82e5bb2320 in ?? ()
#6 0x0000614000022498 in ?? ()
#7 0x00007f82e5bb23a0 in ?? ()
#8 0x00007f82f445c25d in ?? ()
#9 0x3fb9742cd62bc000 in ?? ()
#10 0x000061a00000f680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f680 in ?? ()
#13 0x00000000a0000000 in ?? ()
#14 0x00007f8200000000 in ?? ()
#15 0x41d9e1892f6ec17e in ?? ()
#16 0x00000ff0dcb6e474 in ?? ()
#17 0x00007f82e5bb2380 in ?? ()
#18 0x00007f82f4460ba3 in ?? ()
#19 0x00007f82e5bb2350 in ?? ()
#20 0x3fb9742cd62bc000 in ?? ()
#21 0x00000000e5bb23a0 in ?? ()
#22 0x000061a00000f680 in ?? ()
#23 0x0000614000022498 in ?? ()
#24 0x3fb9742cd62bc000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 7629):
#0 0x00007f82f1dfca47 in ?? ()
#1 0x00007f82e63b32e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f82e63b32d0 in ?? ()
#4 0x00007f82e63b34e0 in ?? ()
#5 0x00007f82e63b3320 in ?? ()
#6 0x0000614000022298 in ?? ()
#7 0x00007f82e63b33a0 in ?? ()
#8 0x00007f82f445c25d in ?? ()
#9 0x3fb96ac24d909000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x0000000094000000 in ?? ()
#14 0x00007f8200000000 in ?? ()
#15 0x41d9e1892f6ec17f in ?? ()
#16 0x00000ff0dcc6e674 in ?? ()
#17 0x00007f82e63b3380 in ?? ()
#18 0x00007f82f4460ba3 in ?? ()
#19 0x00007f82e63b3350 in ?? ()
#20 0x3fb96ac24d909000 in ?? ()
#21 0x00000000e63b33a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000022298 in ?? ()
#24 0x3fb96ac24d909000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 7628):
#0 0x00007f82f1dfca47 in ?? ()
#1 0x00007f82e6bb42e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f82e6bb42d0 in ?? ()
#4 0x00007f82e6bb44e0 in ?? ()
#5 0x00007f82e6bb4320 in ?? ()
#6 0x0000614000022098 in ?? ()
#7 0x00007f82e6bb43a0 in ?? ()
#8 0x00007f82f445c25d in ?? ()
#9 0x3fa1c27dc70b4000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x00000000a0800000 in ?? ()
#14 0x00007f8200000000 in ?? ()
#15 0x41d9e1892f6ec17f in ?? ()
#16 0x00000ff0dcd6e874 in ?? ()
#17 0x00007f82e6bb4380 in ?? ()
#18 0x00007f82f4460ba3 in ?? ()
#19 0x00007f82e6bb4350 in ?? ()
#20 0x3fa1c27dc70b4000 in ?? ()
#21 0x00000000e6bb43a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000022098 in ?? ()
#24 0x3fa1c27dc70b4000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 7627):
#0 0x00007f82f1dfca47 in ?? ()
#1 0x00007f82e8bad2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f82e8bad2d0 in ?? ()
#4 0x00007f82e8bad4e0 in ?? ()
#5 0x00007f82e8bad320 in ?? ()
#6 0x0000614000021e98 in ?? ()
#7 0x00007f82e8bad3a0 in ?? ()
#8 0x00007f82f445c25d in ?? ()
#9 0x3fb978a786987000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x00000000d2800000 in ?? ()
#14 0x00007f8200000000 in ?? ()
#15 0x41d9e1892f6ec180 in ?? ()
#16 0x00000ff0dd16da74 in ?? ()
#17 0x00007f82e8bad380 in ?? ()
#18 0x00007f82f4460ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 7624):
#0 0x00007f82f1defcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f82ea5b7788 in ?? ()
#3 0x00006020000189d0 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 7623):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x00007f82e9db6080 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 7622):
#0 0x00007f82f908f9e2 in ?? ()
#1 0x00007f82e95b3b60 in ?? ()
#2 0x00007f82e95b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f82e95b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f82e95b3c10 in ?? ()
#8 0x0000611000088950 in ?? ()
#9 0x00007f82e95b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f82fe9b3f0c in ?? ()
#12 0x00007f82fe9a37b9 in ?? ()
#13 0x00007f82fe9a7da6 in ?? ()
#14 0x00007f82fe9ac7e0 in ?? ()
#15 0x00007f82fe9ac609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f82f5ca39be in ?? ()
#18 0x00007f82f90856db in ?? ()
#19 0x00000ff0dd2ae67c in ?? ()
#20 0x00007f82e95b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 7616):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x00007f82eb5b8c40 in ?? ()
#2 0x0000000000000019 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f82eb5b8c30 in ?? ()
#6 0x0000000000000032 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 7615):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 7614):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 7613):
#0 0x00007f82f908bfb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f82ecdbc410 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 7612):
#0 0x00007f82f908fd50 in ?? ()
#1 0x0000000000000000 in ?? ()
I20250114 08:56:27.809330 8133 consensus_queue.cc:1035] T b9e44cce5bbe4d258ea9fa8ac8741fb4 P 96e3c2684cd4457685c687fcf04fce75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38213dd9fed845bf860fcb89a1fb8745" member_type: VOTER last_known_addr { host: "127.7.86.129" port: 41225 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
************************* END STACKS ***************************
I20250114 08:56:27.818681 7514 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 1 with UUID 96e3c2684cd4457685c687fcf04fce75 and pid 7742
************************ BEGIN STACKS **************************
[New LWP 7743]
[New LWP 7744]
[New LWP 7745]
[New LWP 7746]
[New LWP 7752]
[New LWP 7753]
[New LWP 7754]
[New LWP 7757]
[New LWP 7758]
[New LWP 7759]
[New LWP 7760]
[New LWP 7761]
[New LWP 7762]
[New LWP 7763]
[New LWP 7764]
[New LWP 7765]
[New LWP 7766]
[New LWP 7767]
[New LWP 7768]
[New LWP 7769]
[New LWP 7770]
[New LWP 7771]
[New LWP 7772]
[New LWP 7773]
[New LWP 7774]
[New LWP 7775]
[New LWP 7776]
[New LWP 7777]
[New LWP 7778]
[New LWP 7779]
[New LWP 7780]
[New LWP 7781]
[New LWP 7782]
[New LWP 7783]
[New LWP 7784]
[New LWP 7785]
[New LWP 7786]
[New LWP 7787]
[New LWP 7788]
[New LWP 7789]
[New LWP 7790]
[New LWP 7791]
[New LWP 7792]
[New LWP 7793]
[New LWP 7794]
[New LWP 7795]
[New LWP 7796]
[New LWP 7797]
[New LWP 7798]
[New LWP 7799]
[New LWP 7800]
[New LWP 7801]
[New LWP 7802]
[New LWP 7803]
[New LWP 7804]
[New LWP 7805]
[New LWP 7806]
[New LWP 7807]
[New LWP 7808]
[New LWP 7809]
[New LWP 7810]
[New LWP 7811]
[New LWP 7812]
[New LWP 7813]
[New LWP 7814]
[New LWP 7815]
[New LWP 7816]
[New LWP 7817]
[New LWP 7818]
[New LWP 7819]
[New LWP 7820]
[New LWP 7821]
[New LWP 7822]
[New LWP 7823]
[New LWP 7824]
[New LWP 7825]
[New LWP 7826]
[New LWP 7827]
[New LWP 7828]
[New LWP 7829]
[New LWP 7830]
[New LWP 7831]
[New LWP 7832]
[New LWP 7833]
[New LWP 7834]
[New LWP 7835]
[New LWP 7836]
[New LWP 7837]
[New LWP 7838]
[New LWP 7839]
[New LWP 7840]
[New LWP 7841]
[New LWP 7842]
[New LWP 7843]
[New LWP 7844]
[New LWP 7845]
[New LWP 7846]
[New LWP 7847]
[New LWP 7848]
[New LWP 7849]
[New LWP 7850]
[New LWP 7851]
[New LWP 7852]
[New LWP 7853]
[New LWP 7854]
[New LWP 7855]
[New LWP 7856]
[New LWP 7857]
[New LWP 7858]
[New LWP 7859]
[New LWP 7860]
[New LWP 7861]
[New LWP 7862]
[New LWP 7863]
[New LWP 7864]
[New LWP 7865]
[New LWP 7866]
[New LWP 7867]
[New LWP 7868]
[New LWP 7869]
[New LWP 7870]
[New LWP 8105]
[New LWP 8123]
[New LWP 8133]
[New LWP 8144]
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
0x00007f78034cbd50 in ?? ()
Id Target Id Frame
* 1 LWP 7742 "kudu" 0x00007f78034cbd50 in ?? ()
2 LWP 7743 "kudu" 0x00007f78034c7fb9 in ?? ()
3 LWP 7744 "kudu" 0x00007f78034c7fb9 in ?? ()
4 LWP 7745 "kudu" 0x00007f78034c7fb9 in ?? ()
5 LWP 7746 "kernel-watcher-" 0x00007f78034c7fb9 in ?? ()
6 LWP 7752 "ntp client-7752" 0x00007f78034cb9e2 in ?? ()
7 LWP 7753 "file cache-evic" 0x00007f78034c7fb9 in ?? ()
8 LWP 7754 "sq_acceptor" 0x00007f77fc22bcb9 in ?? ()
9 LWP 7757 "rpc reactor-775" 0x00007f77fc238a47 in ?? ()
10 LWP 7758 "rpc reactor-775" 0x00007f77fc238a47 in ?? ()
11 LWP 7759 "rpc reactor-775" 0x00007f77fc238a47 in ?? ()
12 LWP 7760 "rpc reactor-776" 0x00007f77fc238a47 in ?? ()
13 LWP 7761 "MaintenanceMgr " 0x00007f78034c7ad3 in ?? ()
14 LWP 7762 "txn-status-mana" 0x00007f78034c7fb9 in ?? ()
15 LWP 7763 "collect_and_rem" 0x00007f78034c7fb9 in ?? ()
16 LWP 7764 "tc-session-exp-" 0x00007f78034c7fb9 in ?? ()
17 LWP 7765 "rpc worker-7765" 0x00007f78034c7ad3 in ?? ()
18 LWP 7766 "rpc worker-7766" 0x00007f78034c7ad3 in ?? ()
19 LWP 7767 "rpc worker-7767" 0x00007f78034c7ad3 in ?? ()
20 LWP 7768 "rpc worker-7768" 0x00007f78034c7ad3 in ?? ()
21 LWP 7769 "rpc worker-7769" 0x00007f78034c7ad3 in ?? ()
22 LWP 7770 "rpc worker-7770" 0x00007f78034c7ad3 in ?? ()
23 LWP 7771 "rpc worker-7771" 0x00007f78034c7ad3 in ?? ()
24 LWP 7772 "rpc worker-7772" 0x00007f78034c7ad3 in ?? ()
25 LWP 7773 "rpc worker-7773" 0x00007f78034c7ad3 in ?? ()
26 LWP 7774 "rpc worker-7774" 0x00007f78034c7ad3 in ?? ()
27 LWP 7775 "rpc worker-7775" 0x00007f78034c7ad3 in ?? ()
28 LWP 7776 "rpc worker-7776" 0x00007f78034c7ad3 in ?? ()
29 LWP 7777 "rpc worker-7777" 0x00007f78034c7ad3 in ?? ()
30 LWP 7778 "rpc worker-7778" 0x00007f78034c7ad3 in ?? ()
31 LWP 7779 "rpc worker-7779" 0x00007f78034c7ad3 in ?? ()
32 LWP 7780 "rpc worker-7780" 0x00007f78034c7ad3 in ?? ()
33 LWP 7781 "rpc worker-7781" 0x00007f78034c7ad3 in ?? ()
34 LWP 7782 "rpc worker-7782" 0x00007f78034c7ad3 in ?? ()
35 LWP 7783 "rpc worker-7783" 0x00007f78034c7ad3 in ?? ()
36 LWP 7784 "rpc worker-7784" 0x00007f78034c7ad3 in ?? ()
37 LWP 7785 "rpc worker-7785" 0x00007f78034c7ad3 in ?? ()
38 LWP 7786 "rpc worker-7786" 0x00007f78034c7ad3 in ?? ()
39 LWP 7787 "rpc worker-7787" 0x00007f78034c7ad3 in ?? ()
40 LWP 7788 "rpc worker-7788" 0x00007f78034c7ad3 in ?? ()
41 LWP 7789 "rpc worker-7789" 0x00007f78034c7ad3 in ?? ()
42 LWP 7790 "rpc worker-7790" 0x00007f78034c7ad3 in ?? ()
43 LWP 7791 "rpc worker-7791" 0x00007f78034c7ad3 in ?? ()
44 LWP 7792 "rpc worker-7792" 0x00007f78034c7ad3 in ?? ()
45 LWP 7793 "rpc worker-7793" 0x00007f78034c7ad3 in ?? ()
46 LWP 7794 "rpc worker-7794" 0x00007f78034c7ad3 in ?? ()
47 LWP 7795 "rpc worker-7795" 0x00007f78034c7ad3 in ?? ()
48 LWP 7796 "rpc worker-7796" 0x00007f78034c7ad3 in ?? ()
49 LWP 7797 "rpc worker-7797" 0x00007f78034c7ad3 in ?? ()
50 LWP 7798 "rpc worker-7798" 0x00007f78034c7ad3 in ?? ()
51 LWP 7799 "rpc worker-7799" 0x00007f78034c7ad3 in ?? ()
52 LWP 7800 "rpc worker-7800" 0x00007f78034c7ad3 in ?? ()
53 LWP 7801 "rpc worker-7801" 0x00007f78034c7ad3 in ?? ()
54 LWP 7802 "rpc worker-7802" 0x00007f78034c7ad3 in ?? ()
55 LWP 7803 "rpc worker-7803" 0x00007f78034c7ad3 in ?? ()
56 LWP 7804 "rpc worker-7804" 0x00007f78034c7ad3 in ?? ()
57 LWP 7805 "rpc worker-7805" 0x00007f78034c7ad3 in ?? ()
58 LWP 7806 "rpc worker-7806" 0x00007f78034c7ad3 in ?? ()
59 LWP 7807 "rpc worker-7807" 0x00007f78034c7ad3 in ?? ()
60 LWP 7808 "rpc worker-7808" 0x00007f78034c7ad3 in ?? ()
61 LWP 7809 "rpc worker-7809" 0x00007f78034c7ad3 in ?? ()
62 LWP 7810 "rpc worker-7810" 0x00007f78034c7ad3 in ?? ()
63 LWP 7811 "rpc worker-7811" 0x00007f78034c7ad3 in ?? ()
64 LWP 7812 "rpc worker-7812" 0x00007f78034c7ad3 in ?? ()
65 LWP 7813 "rpc worker-7813" 0x00007f78034c7ad3 in ?? ()
66 LWP 7814 "rpc worker-7814" 0x00007f78034c7ad3 in ?? ()
67 LWP 7815 "rpc worker-7815" 0x00007f78034c7ad3 in ?? ()
68 LWP 7816 "rpc worker-7816" 0x00007f78034c7ad3 in ?? ()
69 LWP 7817 "rpc worker-7817" 0x00007f78034c7ad3 in ?? ()
70 LWP 7818 "rpc worker-7818" 0x00007f78034c7ad3 in ?? ()
71 LWP 7819 "rpc worker-7819" 0x00007f78034c7ad3 in ?? ()
72 LWP 7820 "rpc worker-7820" 0x00007f78034c7ad3 in ?? ()
73 LWP 7821 "rpc worker-7821" 0x00007f78034c7ad3 in ?? ()
74 LWP 7822 "rpc worker-7822" 0x00007f78034c7ad3 in ?? ()
75 LWP 7823 "rpc worker-7823" 0x00007f78034c7ad3 in ?? ()
76 LWP 7824 "rpc worker-7824" 0x00007f78034c7ad3 in ?? ()
77 LWP 7825 "rpc worker-7825" 0x00007f78034c7ad3 in ?? ()
78 LWP 7826 "rpc worker-7826" 0x00007f78034c7ad3 in ?? ()
79 LWP 7827 "rpc worker-7827" 0x00007f78034c7ad3 in ?? ()
80 LWP 7828 "rpc worker-7828" 0x00007f78034c7ad3 in ?? ()
81 LWP 7829 "rpc worker-7829" 0x00007f78034c7ad3 in ?? ()
82 LWP 7830 "rpc worker-7830" 0x00007f78034c7ad3 in ?? ()
83 LWP 7831 "rpc worker-7831" 0x00007f78034c7ad3 in ?? ()
84 LWP 7832 "rpc worker-7832" 0x00007f78034c7ad3 in ?? ()
85 LWP 7833 "rpc worker-7833" 0x00007f78034c7ad3 in ?? ()
86 LWP 7834 "rpc worker-7834" 0x00007f78034c7ad3 in ?? ()
87 LWP 7835 "rpc worker-7835" 0x00007f78034c7ad3 in ?? ()
88 LWP 7836 "rpc worker-7836" 0x00007f78034c7ad3 in ?? ()
89 LWP 7837 "rpc worker-7837" 0x00007f78034c7ad3 in ?? ()
90 LWP 7838 "rpc worker-7838" 0x00007f78034c7ad3 in ?? ()
91 LWP 7839 "rpc worker-7839" 0x00007f78034c7ad3 in ?? ()
92 LWP 7840 "rpc worker-7840" 0x00007f78034c7ad3 in ?? ()
93 LWP 7841 "rpc worker-7841" 0x00007f78034c7ad3 in ?? ()
94 LWP 7842 "rpc worker-7842" 0x00007f78034c7ad3 in ?? ()
95 LWP 7843 "rpc worker-7843" 0x00007f78034c7ad3 in ?? ()
96 LWP 7844 "rpc worker-7844" 0x00007f78034c7ad3 in ?? ()
97 LWP 7845 "rpc worker-7845" 0x00007f78034c7ad3 in ?? ()
98 LWP 7846 "rpc worker-7846" 0x00007f78034c7ad3 in ?? ()
99 LWP 7847 "rpc worker-7847" 0x00007f78034c7ad3 in ?? ()
100 LWP 7848 "rpc worker-7848" 0x00007f78034c7ad3 in ?? ()
101 LWP 7849 "rpc worker-7849" 0x00007f78034c7ad3 in ?? ()
102 LWP 7850 "rpc worker-7850" 0x00007f78034c7ad3 in ?? ()
103 LWP 7851 "rpc worker-7851" 0x00007f78034c7ad3 in ?? ()
104 LWP 7852 "rpc worker-7852" 0x00007f78034c7ad3 in ?? ()
105 LWP 7853 "rpc worker-7853" 0x00007f78034c7ad3 in ?? ()
106 LWP 7854 "rpc worker-7854" 0x00007f78034c7ad3 in ?? ()
107 LWP 7855 "rpc worker-7855" 0x00007f78034c7ad3 in ?? ()
108 LWP 7856 "rpc worker-7856" 0x00007f78034c7ad3 in ?? ()
109 LWP 7857 "rpc worker-7857" 0x00007f78034c7ad3 in ?? ()
110 LWP 7858 "rpc worker-7858" 0x00007f78034c7ad3 in ?? ()
111 LWP 7859 "rpc worker-7859" 0x00007f78034c7ad3 in ?? ()
112 LWP 7860 "rpc worker-7860" 0x00007f78034c7ad3 in ?? ()
113 LWP 7861 "rpc worker-7861" 0x00007f78034c7ad3 in ?? ()
114 LWP 7862 "rpc worker-7862" 0x00007f78034c7ad3 in ?? ()
115 LWP 7863 "rpc worker-7863" 0x00007f78034c7ad3 in ?? ()
116 LWP 7864 "rpc worker-7864" 0x00007f78034c7ad3 in ?? ()
117 LWP 7865 "diag-logger-786" 0x00007f78034c7fb9 in ?? ()
118 LWP 7866 "result-tracker-" 0x00007f78034c7fb9 in ?? ()
119 LWP 7867 "excess-log-dele" 0x00007f78034c7fb9 in ?? ()
120 LWP 7868 "acceptor-7868" 0x00007f77fc23a0c7 in ?? ()
121 LWP 7869 "heartbeat-7869" 0x00007f78034c7fb9 in ?? ()
122 LWP 7870 "maintenance_sch" 0x00007f78034c7fb9 in ?? ()
123 LWP 8105 "raft [worker]-8" 0x00007f78034c7fb9 in ?? ()
124 LWP 8123 "wal-append [wor" 0x00007f78034c7fb9 in ?? ()
125 LWP 8133 "raft [worker]-8" 0x00007f78034c7fb9 in ?? ()
126 LWP 8144 "raft [worker]-8" 0x00007f78034c7fb9 in ?? ()
Thread 126 (LWP 8144):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000003 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007f77b2bb0b24 in ?? ()
#5 0x00007f77b2bb07b0 in ?? ()
#6 0x0000000000000007 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f77b2bb07d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 125 (LWP 8133):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000008 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007f77b4432b20 in ?? ()
#5 0x00007f77b44327b0 in ?? ()
#6 0x0000000000000010 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 124 (LWP 8123):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x00006020001d00f8 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x000061a0001501b4 in ?? ()
#5 0x00007f77b33da090 in ?? ()
#6 0x0000000000000003 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f77b33da0b0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 123 (LWP 8105):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000033 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007f77b1324b24 in ?? ()
#5 0x00007f77b13247b0 in ?? ()
#6 0x0000000000000067 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f77b13247d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 122 (LWP 7870):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f77b858a550 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 7869):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x9fa35ef76db3e6dd in ?? ()
#2 0x000000000000000f in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007f77b8da25b0 in ?? ()
#6 0x000000000000001f in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f77b8da25d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f77b8da2650 in ?? ()
#12 0x00000feef71b44cc in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 7868):
#0 0x00007f77fc23a0c7 in ?? ()
#1 0x00006140000222a8 in ?? ()
#2 0x00007f77b95c9d50 in ?? ()
#3 0x00007f77b95c9d80 in ?? ()
#4 0x00007f77b95c9e80 in ?? ()
#5 0x00007f77b95c9d70 in ?? ()
#6 0x00007f77b95c9de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007f77b95c9f00 in ?? ()
#11 0x00007f77b95c963c in ?? ()
#12 0x00000032b95c95b0 in ?? ()
#13 0x00007f77b8dcd000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 7867):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x00007f77b9de3f00 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 7866):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x00007f77ba5fc0c0 in ?? ()
#2 0x0000000000000005 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000879b0 in ?? ()
#5 0x00007f77ba5fc0b0 in ?? ()
#6 0x000000000000000a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 7865):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 7864):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 115 (LWP 7863):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 114 (LWP 7862):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 7861):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 7860):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 7859):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 7858):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 7857):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 7856):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 7855):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 7854):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 7853):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 7852):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 7851):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 7850):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 7849):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 7848):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 7847):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 7846):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 7845):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 7844):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 7843):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 7842):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 7841):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 7840):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 7839):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 7838):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 7837):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 7836):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 7835):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 7834):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 7833):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 7832):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 7831):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 7830):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 7829):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 7828):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 7827):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 7826):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 7825):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 7824):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000048 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007f77cf9ece50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77cf9ece70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 7823):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x00000000000000cc in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007f77d0204e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77d0204e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 7822):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x00000000000000a4 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a0088 in ?? ()
#4 0x00007f77d0a1ce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77d0a1ce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 73 (LWP 7821):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 7820):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 7819):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 7818):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 7817):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 7816):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 7815):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 7814):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 7813):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 7812):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 7811):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 7810):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 7809):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 7808):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 7807):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 7806):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 7805):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 7804):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007f77d9bcde50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77d9bcde70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 7803):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f77da3e5e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77da3e5e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f78034c7770 in ?? ()
#10 0x00007f77da3e5e70 in ?? ()
#11 0x00000feefb47cbd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 54 (LWP 7802):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00011d048 in ?? ()
#4 0x00007f77dabfde50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77dabfde70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 53 (LWP 7801):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00011684c in ?? ()
#4 0x00007f77db415e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77db415e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d000116800 in ?? ()
#9 0x00007f78034c7770 in ?? ()
#10 0x00007f77db415e70 in ?? ()
#11 0x00000feefb682bd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 52 (LWP 7800):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 7799):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 7798):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 7797):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 7796):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 7795):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 7794):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 7793):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 7792):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 7791):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 7790):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 7789):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 7788):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 7787):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 7786):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 7785):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 7784):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000091 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f77e3daee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77e3daee70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f78034c7770 in ?? ()
#10 0x00007f77e3daee70 in ?? ()
#11 0x00000feefc7b5dd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 7783):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x000000000000000b in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f77e45c6e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77e45c6e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f78034c7770 in ?? ()
#10 0x00007f77e45c6e70 in ?? ()
#11 0x00000feefc8b8dd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 34 (LWP 7782):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x000000000000002c in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000967f8 in ?? ()
#4 0x00007f77e4ddee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77e4ddee70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 33 (LWP 7781):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000074 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008fff8 in ?? ()
#4 0x00007f77e55f6e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77e55f6e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 32 (LWP 7780):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008d00c in ?? ()
#4 0x00007f77e5e0ee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77e5e0ee70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00008cfc0 in ?? ()
#9 0x00007f78034c7770 in ?? ()
#10 0x00007f77e5e0ee70 in ?? ()
#11 0x00000feefcbc1dd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 31 (LWP 7779):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008680c in ?? ()
#4 0x00007f77e6626e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77e6626e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000867c0 in ?? ()
#9 0x00007f78034c7770 in ?? ()
#10 0x00007f77e6626e70 in ?? ()
#11 0x00000feefccc4dd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 30 (LWP 7778):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000074 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000080008 in ?? ()
#4 0x00007f77e6e3ee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77e6e3ee70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 29 (LWP 7777):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x000000000000000a in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000076818 in ?? ()
#4 0x00007f77e7656e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f77e7656e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 28 (LWP 7776):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 7775):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 7774):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 7773):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 7772):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 7771):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 7770):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 7769):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 7768):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 7767):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 7766):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 7765):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 7764):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x00007f77edfb6c80 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 15 (LWP 7763):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 7762):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x00007f77eeff41c0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 13 (LWP 7761):
#0 0x00007f78034c7ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 7760):
#0 0x00007f77fc238a47 in ?? ()
#1 0x00007f77f00332e0 in ?? ()
#2 0x000061a00000f680 in ?? ()
#3 0x00007f77f00332d0 in ?? ()
#4 0x00007f77f00334e0 in ?? ()
#5 0x00007f77f0033320 in ?? ()
#6 0x0000614000022498 in ?? ()
#7 0x00007f77f00333a0 in ?? ()
#8 0x00007f77fe89825d in ?? ()
#9 0x3fb980e6f4866000 in ?? ()
#10 0x000061a00000f680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f680 in ?? ()
#13 0x00000000db800000 in ?? ()
#14 0x00007f7700000000 in ?? ()
#15 0x41d9e1892f6ec182 in ?? ()
#16 0x00000fef7dffe674 in ?? ()
#17 0x00007f77f0033380 in ?? ()
#18 0x00007f77fe89cba3 in ?? ()
#19 0x00007f77f0033350 in ?? ()
#20 0x3fb980e6f4866000 in ?? ()
#21 0x00000000f00333a0 in ?? ()
#22 0x000061a00000f680 in ?? ()
#23 0x0000614000022498 in ?? ()
#24 0x3fb980e6f4866000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 7759):
#0 0x00007f77fc238a47 in ?? ()
#1 0x00007f77f084a2e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f77f084a2d0 in ?? ()
#4 0x00007f77f084a4e0 in ?? ()
#5 0x00007f77f084a320 in ?? ()
#6 0x0000614000022298 in ?? ()
#7 0x00007f77f084a3a0 in ?? ()
#8 0x00007f77fe89825d in ?? ()
#9 0x3fb958dcdd5ee000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000bc000000 in ?? ()
#14 0x00007f7700000000 in ?? ()
#15 0x41d9e1892f6ec181 in ?? ()
#16 0x00000fef7e101474 in ?? ()
#17 0x00007f77f084a380 in ?? ()
#18 0x00007f77fe89cba3 in ?? ()
#19 0x00007f77f084a350 in ?? ()
#20 0x3fb958dcdd5ee000 in ?? ()
#21 0x00000000f084a3a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000022298 in ?? ()
#24 0x3fb958dcdd5ee000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 7758):
#0 0x00007f77fc238a47 in ?? ()
#1 0x00007f77f10612e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f77f10612d0 in ?? ()
#4 0x00007f77f10614e0 in ?? ()
#5 0x00007f77f1061320 in ?? ()
#6 0x0000614000022098 in ?? ()
#7 0x00007f77f10613a0 in ?? ()
#8 0x00007f77fe89825d in ?? ()
#9 0x3fb97d68d489b000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x000000006a000000 in ?? ()
#14 0x00007f7700000000 in ?? ()
#15 0x41d9e1892f6ec182 in ?? ()
#16 0x00000fef7e204274 in ?? ()
#17 0x00007f77f1061380 in ?? ()
#18 0x00007f77fe89cba3 in ?? ()
#19 0x00007f77f1061350 in ?? ()
#20 0x3fb97d68d489b000 in ?? ()
#21 0x00000000f10613a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000022098 in ?? ()
#24 0x3fb97d68d489b000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 7757):
#0 0x00007f77fc238a47 in ?? ()
#1 0x00007f77f30872e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f77f30872d0 in ?? ()
#4 0x00007f77f30874e0 in ?? ()
#5 0x00007f77f3087320 in ?? ()
#6 0x0000614000021e98 in ?? ()
#7 0x00007f77f30873a0 in ?? ()
#8 0x00007f77fe89825d in ?? ()
#9 0x3fb951a5a4962000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000079000000 in ?? ()
#14 0x00007f7700000000 in ?? ()
#15 0x41d9e1892f6ec185 in ?? ()
#16 0x00000fef7e608e74 in ?? ()
#17 0x00007f77f3087380 in ?? ()
#18 0x00007f77fe89cba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 7754):
#0 0x00007f77fc22bcb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f77f4ab7788 in ?? ()
#3 0x0000602000016790 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 7753):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x00007f77f42b6080 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 7752):
#0 0x00007f78034cb9e2 in ?? ()
#1 0x00007f77f3ab3b60 in ?? ()
#2 0x00007f77f3ab3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f77f3ab3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f77f3ab3c10 in ?? ()
#8 0x0000611000088950 in ?? ()
#9 0x00007f77f3ab3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f7808deff0c in ?? ()
#12 0x00007f7808ddf7b9 in ?? ()
#13 0x00007f7808de3da6 in ?? ()
#14 0x00007f7808de87e0 in ?? ()
#15 0x00007f7808de8609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f78000df9be in ?? ()
#18 0x00007f78034c16db in ?? ()
#19 0x00000fef7e74e67c in ?? ()
#20 0x00007f77f3ab3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 7746):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x00007f77f5ab8c40 in ?? ()
#2 0x000000000000001a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f77f5ab8c30 in ?? ()
#6 0x0000000000000034 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 7745):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 7744):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 7743):
#0 0x00007f78034c7fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f77f72bc410 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 7742):
#0 0x00007f78034cbd50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250114 08:56:28.382802 7514 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 2 with UUID 15e6413d6f5047b8acc75dc385a3905e and pid 7872
************************ BEGIN STACKS **************************
[New LWP 7873]
[New LWP 7874]
[New LWP 7875]
[New LWP 7876]
[New LWP 7882]
[New LWP 7883]
[New LWP 7884]
[New LWP 7887]
[New LWP 7888]
[New LWP 7889]
[New LWP 7890]
[New LWP 7891]
[New LWP 7892]
[New LWP 7893]
[New LWP 7894]
[New LWP 7895]
[New LWP 7896]
[New LWP 7897]
[New LWP 7898]
[New LWP 7899]
[New LWP 7900]
[New LWP 7901]
[New LWP 7902]
[New LWP 7903]
[New LWP 7904]
[New LWP 7905]
[New LWP 7906]
[New LWP 7907]
[New LWP 7908]
[New LWP 7909]
[New LWP 7910]
[New LWP 7911]
[New LWP 7912]
[New LWP 7913]
[New LWP 7914]
[New LWP 7915]
[New LWP 7916]
[New LWP 7917]
[New LWP 7918]
[New LWP 7919]
[New LWP 7920]
[New LWP 7921]
[New LWP 7922]
[New LWP 7923]
[New LWP 7924]
[New LWP 7925]
[New LWP 7926]
[New LWP 7927]
[New LWP 7928]
[New LWP 7929]
[New LWP 7930]
[New LWP 7931]
[New LWP 7932]
[New LWP 7933]
[New LWP 7934]
[New LWP 7935]
[New LWP 7936]
[New LWP 7937]
[New LWP 7938]
[New LWP 7939]
[New LWP 7940]
[New LWP 7941]
[New LWP 7942]
[New LWP 7943]
[New LWP 7944]
[New LWP 7945]
[New LWP 7946]
[New LWP 7947]
[New LWP 7948]
[New LWP 7949]
[New LWP 7950]
[New LWP 7951]
[New LWP 7952]
[New LWP 7953]
[New LWP 7954]
[New LWP 7955]
[New LWP 7956]
[New LWP 7957]
[New LWP 7958]
[New LWP 7959]
[New LWP 7960]
[New LWP 7961]
[New LWP 7962]
[New LWP 7963]
[New LWP 7964]
[New LWP 7965]
[New LWP 7966]
[New LWP 7967]
[New LWP 7968]
[New LWP 7969]
[New LWP 7970]
[New LWP 7971]
[New LWP 7972]
[New LWP 7973]
[New LWP 7974]
[New LWP 7975]
[New LWP 7976]
[New LWP 7977]
[New LWP 7978]
[New LWP 7979]
[New LWP 7980]
[New LWP 7981]
[New LWP 7982]
[New LWP 7983]
[New LWP 7984]
[New LWP 7985]
[New LWP 7986]
[New LWP 7987]
[New LWP 7988]
[New LWP 7989]
[New LWP 7990]
[New LWP 7991]
[New LWP 7992]
[New LWP 7993]
[New LWP 7994]
[New LWP 7995]
[New LWP 7996]
[New LWP 7997]
[New LWP 7998]
[New LWP 7999]
[New LWP 8000]
[New LWP 8086]
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
0x00007f99d2853d50 in ?? ()
Id Target Id Frame
* 1 LWP 7872 "kudu" 0x00007f99d2853d50 in ?? ()
2 LWP 7873 "kudu" 0x00007f99d284ffb9 in ?? ()
3 LWP 7874 "kudu" 0x00007f99d284ffb9 in ?? ()
4 LWP 7875 "kudu" 0x00007f99d284ffb9 in ?? ()
5 LWP 7876 "kernel-watcher-" 0x00007f99d284ffb9 in ?? ()
6 LWP 7882 "ntp client-7882" 0x00007f99d28539e2 in ?? ()
7 LWP 7883 "file cache-evic" 0x00007f99d284ffb9 in ?? ()
8 LWP 7884 "sq_acceptor" 0x00007f99cb5b3cb9 in ?? ()
9 LWP 7887 "rpc reactor-788" 0x00007f99cb5c0a47 in ?? ()
10 LWP 7888 "rpc reactor-788" 0x00007f99cb5c0a47 in ?? ()
11 LWP 7889 "rpc reactor-788" 0x00007f99cb5c0a47 in ?? ()
12 LWP 7890 "rpc reactor-789" 0x00007f99cb5c0a47 in ?? ()
13 LWP 7891 "MaintenanceMgr " 0x00007f99d284fad3 in ?? ()
14 LWP 7892 "txn-status-mana" 0x00007f99d284ffb9 in ?? ()
15 LWP 7893 "collect_and_rem" 0x00007f99d284ffb9 in ?? ()
16 LWP 7894 "tc-session-exp-" 0x00007f99d284ffb9 in ?? ()
17 LWP 7895 "rpc worker-7895" 0x00007f99d284fad3 in ?? ()
18 LWP 7896 "rpc worker-7896" 0x00007f99d284fad3 in ?? ()
19 LWP 7897 "rpc worker-7897" 0x00007f99d284fad3 in ?? ()
20 LWP 7898 "rpc worker-7898" 0x00007f99d284fad3 in ?? ()
21 LWP 7899 "rpc worker-7899" 0x00007f99d284fad3 in ?? ()
22 LWP 7900 "rpc worker-7900" 0x00007f99d284fad3 in ?? ()
23 LWP 7901 "rpc worker-7901" 0x00007f99d284fad3 in ?? ()
24 LWP 7902 "rpc worker-7902" 0x00007f99d284fad3 in ?? ()
25 LWP 7903 "rpc worker-7903" 0x00007f99d284fad3 in ?? ()
26 LWP 7904 "rpc worker-7904" 0x00007f99d284fad3 in ?? ()
27 LWP 7905 "rpc worker-7905" 0x00007f99d284fad3 in ?? ()
28 LWP 7906 "rpc worker-7906" 0x00007f99d284fad3 in ?? ()
29 LWP 7907 "rpc worker-7907" 0x00007f99d284fad3 in ?? ()
30 LWP 7908 "rpc worker-7908" 0x00007f99d284fad3 in ?? ()
31 LWP 7909 "rpc worker-7909" 0x00007f99d284fad3 in ?? ()
32 LWP 7910 "rpc worker-7910" 0x00007f99d284fad3 in ?? ()
33 LWP 7911 "rpc worker-7911" 0x00007f99d284fad3 in ?? ()
34 LWP 7912 "rpc worker-7912" 0x00007f99d284fad3 in ?? ()
35 LWP 7913 "rpc worker-7913" 0x00007f99d284fad3 in ?? ()
36 LWP 7914 "rpc worker-7914" 0x00007f99d284fad3 in ?? ()
37 LWP 7915 "rpc worker-7915" 0x00007f99d284fad3 in ?? ()
38 LWP 7916 "rpc worker-7916" 0x00007f99d284fad3 in ?? ()
39 LWP 7917 "rpc worker-7917" 0x00007f99d284fad3 in ?? ()
40 LWP 7918 "rpc worker-7918" 0x00007f99d284fad3 in ?? ()
41 LWP 7919 "rpc worker-7919" 0x00007f99d284fad3 in ?? ()
42 LWP 7920 "rpc worker-7920" 0x00007f99d284fad3 in ?? ()
43 LWP 7921 "rpc worker-7921" 0x00007f99d284fad3 in ?? ()
44 LWP 7922 "rpc worker-7922" 0x00007f99d284fad3 in ?? ()
45 LWP 7923 "rpc worker-7923" 0x00007f99d284fad3 in ?? ()
46 LWP 7924 "rpc worker-7924" 0x00007f99d284fad3 in ?? ()
47 LWP 7925 "rpc worker-7925" 0x00007f99d284fad3 in ?? ()
48 LWP 7926 "rpc worker-7926" 0x00007f99d284fad3 in ?? ()
49 LWP 7927 "rpc worker-7927" 0x00007f99d284fad3 in ?? ()
50 LWP 7928 "rpc worker-7928" 0x00007f99d284fad3 in ?? ()
51 LWP 7929 "rpc worker-7929" 0x00007f99d284fad3 in ?? ()
52 LWP 7930 "rpc worker-7930" 0x00007f99d284fad3 in ?? ()
53 LWP 7931 "rpc worker-7931" 0x00007f99d284fad3 in ?? ()
54 LWP 7932 "rpc worker-7932" 0x00007f99d284fad3 in ?? ()
55 LWP 7933 "rpc worker-7933" 0x00007f99d284fad3 in ?? ()
56 LWP 7934 "rpc worker-7934" 0x00007f99d284fad3 in ?? ()
57 LWP 7935 "rpc worker-7935" 0x00007f99d284fad3 in ?? ()
58 LWP 7936 "rpc worker-7936" 0x00007f99d284fad3 in ?? ()
59 LWP 7937 "rpc worker-7937" 0x00007f99d284fad3 in ?? ()
60 LWP 7938 "rpc worker-7938" 0x00007f99d284fad3 in ?? ()
61 LWP 7939 "rpc worker-7939" 0x00007f99d284fad3 in ?? ()
62 LWP 7940 "rpc worker-7940" 0x00007f99d284fad3 in ?? ()
63 LWP 7941 "rpc worker-7941" 0x00007f99d284fad3 in ?? ()
64 LWP 7942 "rpc worker-7942" 0x00007f99d284fad3 in ?? ()
65 LWP 7943 "rpc worker-7943" 0x00007f99d284fad3 in ?? ()
66 LWP 7944 "rpc worker-7944" 0x00007f99d284fad3 in ?? ()
67 LWP 7945 "rpc worker-7945" 0x00007f99d284fad3 in ?? ()
68 LWP 7946 "rpc worker-7946" 0x00007f99d284fad3 in ?? ()
69 LWP 7947 "rpc worker-7947" 0x00007f99d284fad3 in ?? ()
70 LWP 7948 "rpc worker-7948" 0x00007f99d284fad3 in ?? ()
71 LWP 7949 "rpc worker-7949" 0x00007f99d284fad3 in ?? ()
72 LWP 7950 "rpc worker-7950" 0x00007f99d284fad3 in ?? ()
73 LWP 7951 "rpc worker-7951" 0x00007f99d284fad3 in ?? ()
74 LWP 7952 "rpc worker-7952" 0x00007f99d284fad3 in ?? ()
75 LWP 7953 "rpc worker-7953" 0x00007f99d284fad3 in ?? ()
76 LWP 7954 "rpc worker-7954" 0x00007f99d284fad3 in ?? ()
77 LWP 7955 "rpc worker-7955" 0x00007f99d284fad3 in ?? ()
78 LWP 7956 "rpc worker-7956" 0x00007f99d284fad3 in ?? ()
79 LWP 7957 "rpc worker-7957" 0x00007f99d284fad3 in ?? ()
80 LWP 7958 "rpc worker-7958" 0x00007f99d284fad3 in ?? ()
81 LWP 7959 "rpc worker-7959" 0x00007f99d284fad3 in ?? ()
82 LWP 7960 "rpc worker-7960" 0x00007f99d284fad3 in ?? ()
83 LWP 7961 "rpc worker-7961" 0x00007f99d284fad3 in ?? ()
84 LWP 7962 "rpc worker-7962" 0x00007f99d284fad3 in ?? ()
85 LWP 7963 "rpc worker-7963" 0x00007f99d284fad3 in ?? ()
86 LWP 7964 "rpc worker-7964" 0x00007f99d284fad3 in ?? ()
87 LWP 7965 "rpc worker-7965" 0x00007f99d284fad3 in ?? ()
88 LWP 7966 "rpc worker-7966" 0x00007f99d284fad3 in ?? ()
89 LWP 7967 "rpc worker-7967" 0x00007f99d284fad3 in ?? ()
90 LWP 7968 "rpc worker-7968" 0x00007f99d284fad3 in ?? ()
91 LWP 7969 "rpc worker-7969" 0x00007f99d284fad3 in ?? ()
92 LWP 7970 "rpc worker-7970" 0x00007f99d284fad3 in ?? ()
93 LWP 7971 "rpc worker-7971" 0x00007f99d284fad3 in ?? ()
94 LWP 7972 "rpc worker-7972" 0x00007f99d284fad3 in ?? ()
95 LWP 7973 "rpc worker-7973" 0x00007f99d284fad3 in ?? ()
96 LWP 7974 "rpc worker-7974" 0x00007f99d284fad3 in ?? ()
97 LWP 7975 "rpc worker-7975" 0x00007f99d284fad3 in ?? ()
98 LWP 7976 "rpc worker-7976" 0x00007f99d284fad3 in ?? ()
99 LWP 7977 "rpc worker-7977" 0x00007f99d284fad3 in ?? ()
100 LWP 7978 "rpc worker-7978" 0x00007f99d284fad3 in ?? ()
101 LWP 7979 "rpc worker-7979" 0x00007f99d284fad3 in ?? ()
102 LWP 7980 "rpc worker-7980" 0x00007f99d284fad3 in ?? ()
103 LWP 7981 "rpc worker-7981" 0x00007f99d284fad3 in ?? ()
104 LWP 7982 "rpc worker-7982" 0x00007f99d284fad3 in ?? ()
105 LWP 7983 "rpc worker-7983" 0x00007f99d284fad3 in ?? ()
106 LWP 7984 "rpc worker-7984" 0x00007f99d284fad3 in ?? ()
107 LWP 7985 "rpc worker-7985" 0x00007f99d284fad3 in ?? ()
108 LWP 7986 "rpc worker-7986" 0x00007f99d284fad3 in ?? ()
109 LWP 7987 "rpc worker-7987" 0x00007f99d284fad3 in ?? ()
110 LWP 7988 "rpc worker-7988" 0x00007f99d284fad3 in ?? ()
111 LWP 7989 "rpc worker-7989" 0x00007f99d284fad3 in ?? ()
112 LWP 7990 "rpc worker-7990" 0x00007f99d284fad3 in ?? ()
113 LWP 7991 "rpc worker-7991" 0x00007f99d284fad3 in ?? ()
114 LWP 7992 "rpc worker-7992" 0x00007f99d284fad3 in ?? ()
115 LWP 7993 "rpc worker-7993" 0x00007f99d284fad3 in ?? ()
116 LWP 7994 "rpc worker-7994" 0x00007f99d284fad3 in ?? ()
117 LWP 7995 "diag-logger-799" 0x00007f99d284ffb9 in ?? ()
118 LWP 7996 "result-tracker-" 0x00007f99d284ffb9 in ?? ()
119 LWP 7997 "excess-log-dele" 0x00007f99d284ffb9 in ?? ()
120 LWP 7998 "acceptor-7998" 0x00007f99cb5c20c7 in ?? ()
121 LWP 7999 "heartbeat-7999" 0x00007f99d284ffb9 in ?? ()
122 LWP 8000 "maintenance_sch" 0x00007f99d284ffb9 in ?? ()
123 LWP 8086 "raft [worker]-8" 0x00007f99d284ffb9 in ?? ()
Thread 123 (LWP 8086):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000108 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007f9985840b20 in ?? ()
#5 0x00007f99858407b0 in ?? ()
#6 0x0000000000000210 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 122 (LWP 8000):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000015 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f9987912550 in ?? ()
#6 0x000000000000002a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 7999):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x9fa35ef76db3e6dd in ?? ()
#2 0x0000000000000010 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007f998812a5b0 in ?? ()
#6 0x0000000000000021 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f998812a5d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f998812a650 in ?? ()
#12 0x00000ff3310254cc in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 7998):
#0 0x00007f99cb5c20c7 in ?? ()
#1 0x00006140000224a8 in ?? ()
#2 0x00007f9988951d50 in ?? ()
#3 0x00007f9988951d80 in ?? ()
#4 0x00007f9988951e80 in ?? ()
#5 0x00007f9988951d70 in ?? ()
#6 0x00007f9988951de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007f9988951f00 in ?? ()
#11 0x00007f998895163c in ?? ()
#12 0x00000032889515b0 in ?? ()
#13 0x00007f9988155000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 7997):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x00007f998916bf00 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 7996):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x00007f99899840c0 in ?? ()
#2 0x0000000000000005 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000879b0 in ?? ()
#5 0x00007f99899840b0 in ?? ()
#6 0x000000000000000a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 7995):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 7994):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 115 (LWP 7993):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 114 (LWP 7992):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 7991):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 7990):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 7989):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 7988):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 7987):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 7986):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 7985):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 7984):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 7983):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 7982):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 7981):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 7980):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 7979):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 7978):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 7977):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 7976):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 7975):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 7974):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 7973):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 7972):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 7971):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 7970):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 7969):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 7968):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 7967):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 7966):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 7965):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 7964):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 7963):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 7962):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 7961):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 7960):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 7959):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 7958):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 7957):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 7956):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 7955):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 7954):
#0 0x00007f99d284fad3 in ?? ()
#1 0x00000000000000aa in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007f999ed74e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f999ed74e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 7953):
#0 0x00007f99d284fad3 in ?? ()
#1 0x000000000000011f in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f999f58ce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f999f58ce70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f99d284f770 in ?? ()
#10 0x00007f999f58ce70 in ?? ()
#11 0x00000ff333eb19d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 74 (LWP 7952):
#0 0x00007f99d284fad3 in ?? ()
#1 0x00000000000000a6 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a0088 in ?? ()
#4 0x00007f999fda4e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f999fda4e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 73 (LWP 7951):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 7950):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 7949):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 7948):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 7947):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 7946):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 7945):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 7944):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 7943):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 7942):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 7941):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 7940):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 7939):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 7938):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 7937):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 7936):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 7935):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 7934):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007f99a8f55e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99a8f55e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 7933):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f99a976de50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99a976de70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f99d284f770 in ?? ()
#10 0x00007f99a976de70 in ?? ()
#11 0x00000ff3352edbd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 54 (LWP 7932):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00011d048 in ?? ()
#4 0x00007f99a9f85e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99a9f85e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 53 (LWP 7931):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00011684c in ?? ()
#4 0x00007f99aa79de50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99aa79de70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d000116800 in ?? ()
#9 0x00007f99d284f770 in ?? ()
#10 0x00007f99aa79de70 in ?? ()
#11 0x00000ff3354f3bd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 52 (LWP 7930):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 7929):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 7928):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 7927):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 7926):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 7925):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 7924):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 7923):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 7922):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 7921):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 7920):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 7919):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 7918):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 7917):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 7916):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 7915):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 7914):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000014 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000a67e8 in ?? ()
#4 0x00007f99b3136e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99b3136e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 35 (LWP 7913):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f99b394ee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99b394ee70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f99d284f770 in ?? ()
#10 0x00007f99b394ee70 in ?? ()
#11 0x00000ff336729dd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 34 (LWP 7912):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000047 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000967fc in ?? ()
#4 0x00007f99b4166e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99b4166e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000967b0 in ?? ()
#9 0x00007f99d284f770 in ?? ()
#10 0x00007f99b4166e70 in ?? ()
#11 0x00000ff33682cdd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 7911):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008fffc in ?? ()
#4 0x00007f99b497ee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99b497ee70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00008ffb0 in ?? ()
#9 0x00007f99d284f770 in ?? ()
#10 0x00007f99b497ee70 in ?? ()
#11 0x00000ff33692fdd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 32 (LWP 7910):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000058 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008d008 in ?? ()
#4 0x00007f99b5196e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99b5196e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 31 (LWP 7909):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000069 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008680c in ?? ()
#4 0x00007f99b59aee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99b59aee70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000867c0 in ?? ()
#9 0x00007f99d284f770 in ?? ()
#10 0x00007f99b59aee70 in ?? ()
#11 0x00000ff336b35dd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 30 (LWP 7908):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008000c in ?? ()
#4 0x00007f99b61c6e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99b61c6e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00007ffc0 in ?? ()
#9 0x00007f99d284f770 in ?? ()
#10 0x00007f99b61c6e70 in ?? ()
#11 0x00000ff336c38dd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 29 (LWP 7907):
#0 0x00007f99d284fad3 in ?? ()
#1 0x000000000000009d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00007681c in ?? ()
#4 0x00007f99b69dee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f99b69dee70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000767d0 in ?? ()
#9 0x00007f99d284f770 in ?? ()
#10 0x00007f99b69dee70 in ?? ()
#11 0x00000ff336d3bdd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 28 (LWP 7906):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 7905):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 7904):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 7903):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 7902):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 7901):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 7900):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 7899):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 7898):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 7897):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 7896):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 7895):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 7894):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x00007f99bd34bc80 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 15 (LWP 7893):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 7892):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x00007f99be3891c0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 13 (LWP 7891):
#0 0x00007f99d284fad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 7890):
#0 0x00007f99cb5c0a47 in ?? ()
#1 0x00007f99bf3b22e0 in ?? ()
#2 0x000061a00000f680 in ?? ()
#3 0x00007f99bf3b22d0 in ?? ()
#4 0x00007f99bf3b24e0 in ?? ()
#5 0x00007f99bf3b2320 in ?? ()
#6 0x0000614000022498 in ?? ()
#7 0x00007f99bf3b23a0 in ?? ()
#8 0x00007f99cdc2025d in ?? ()
#9 0x3fb9880e861ae000 in ?? ()
#10 0x000061a00000f680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f680 in ?? ()
#13 0x0000000032000000 in ?? ()
#14 0x00007f9900000000 in ?? ()
#15 0x41d9e1892f6ec180 in ?? ()
#16 0x00000ff3b7e6e474 in ?? ()
#17 0x00007f99bf3b2380 in ?? ()
#18 0x00007f99cdc24ba3 in ?? ()
#19 0x00007f99bf3b2350 in ?? ()
#20 0x3fb9880e861ae000 in ?? ()
#21 0x00000000bf3b23a0 in ?? ()
#22 0x000061a00000f680 in ?? ()
#23 0x0000614000022498 in ?? ()
#24 0x3fb9880e861ae000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 7889):
#0 0x00007f99cb5c0a47 in ?? ()
#1 0x00007f99bfbb32e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f99bfbb32d0 in ?? ()
#4 0x00007f99bfbb34e0 in ?? ()
#5 0x00007f99bfbb3320 in ?? ()
#6 0x0000614000022298 in ?? ()
#7 0x00007f99bfbb33a0 in ?? ()
#8 0x00007f99cdc2025d in ?? ()
#9 0x3fb96fa970a89000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000ae000000 in ?? ()
#14 0x00007f9900000000 in ?? ()
#15 0x41d9e1892f6ec180 in ?? ()
#16 0x00000ff3b7f6e674 in ?? ()
#17 0x00007f99bfbb3380 in ?? ()
#18 0x00007f99cdc24ba3 in ?? ()
#19 0x00007f99bfbb3350 in ?? ()
#20 0x3fb96fa970a89000 in ?? ()
#21 0x00000000bfbb33a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000022298 in ?? ()
#24 0x3fb96fa970a89000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 7888):
#0 0x00007f99cb5c0a47 in ?? ()
#1 0x00007f99c03b42e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f99c03b42d0 in ?? ()
#4 0x00007f99c03b44e0 in ?? ()
#5 0x00007f99c03b4320 in ?? ()
#6 0x0000614000022098 in ?? ()
#7 0x00007f99c03b43a0 in ?? ()
#8 0x00007f99cdc2025d in ?? ()
#9 0x3fb970123f51b000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x00000000ec000000 in ?? ()
#14 0x00007f9900000000 in ?? ()
#15 0x41d9e1892f6ec180 in ?? ()
#16 0x00000ff3b806e874 in ?? ()
#17 0x00007f99c03b4380 in ?? ()
#18 0x00007f99cdc24ba3 in ?? ()
#19 0x00007f99c03b4350 in ?? ()
#20 0x3fb970123f51b000 in ?? ()
#21 0x00000000c03b43a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000022098 in ?? ()
#24 0x3fb970123f51b000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 7887):
#0 0x00007f99cb5c0a47 in ?? ()
#1 0x00007f99c23ad2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f99c23ad2d0 in ?? ()
#4 0x00007f99c23ad4e0 in ?? ()
#5 0x00007f99c23ad320 in ?? ()
#6 0x0000614000021e98 in ?? ()
#7 0x00007f99c23ad3a0 in ?? ()
#8 0x00007f99cdc2025d in ?? ()
#9 0x3fad9b19d4f3a000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000092400000 in ?? ()
#14 0x00007f9900000000 in ?? ()
#15 0x41d9e1892f6ec17f in ?? ()
#16 0x00000ff3b846da74 in ?? ()
#17 0x00007f99c23ad380 in ?? ()
#18 0x00007f99cdc24ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 7884):
#0 0x00007f99cb5b3cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f99c3db7788 in ?? ()
#3 0x0000602000016790 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 7883):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x00007f99c35b6080 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 7882):
#0 0x00007f99d28539e2 in ?? ()
#1 0x00007f99c2db3b60 in ?? ()
#2 0x00007f99c2db3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f99c2db3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f99c2db3c10 in ?? ()
#8 0x0000611000088950 in ?? ()
#9 0x00007f99c2db3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f99d8177f0c in ?? ()
#12 0x00007f99d81677b9 in ?? ()
#13 0x00007f99d816bda6 in ?? ()
#14 0x00007f99d81707e0 in ?? ()
#15 0x00007f99d8170609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f99cf4679be in ?? ()
#18 0x00007f99d28496db in ?? ()
#19 0x00000ff3b85ae67c in ?? ()
#20 0x00007f99c2db3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 7876):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x00007f99c4db8c40 in ?? ()
#2 0x000000000000001a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f99c4db8c30 in ?? ()
#6 0x0000000000000034 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 7875):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 7874):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 7873):
#0 0x00007f99d284ffb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f99c65bc410 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 7872):
#0 0x00007f99d2853d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250114 08:56:28.935896 7514 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu pid 7612
I20250114 08:56:28.937281 7734 generic_service.cc:196] Checking for leaks (request via RPC)
I20250114 08:56:29.050307 7514 external_mini_cluster.cc:1620] Killing /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu with pid 7612
I20250114 08:56:29.071810 7514 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu pid 7742
I20250114 08:56:29.082283 7864 generic_service.cc:196] Checking for leaks (request via RPC)
W20250114 08:56:29.181385 7890 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.86.129:41225: connect: Connection refused (error 111)
W20250114 08:56:29.182685 7890 consensus_peers.cc:487] T 3c6e55ceb8354ec9b789da6a49b5d0cd P 15e6413d6f5047b8acc75dc385a3905e -> Peer 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225): Couldn't send request to peer 38213dd9fed845bf860fcb89a1fb8745. Status: Network error: Client connection negotiation failed: client connection to 127.7.86.129:41225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20250114 08:56:29.207912 7760 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.86.129:41225: connect: Connection refused (error 111)
W20250114 08:56:29.209164 7760 consensus_peers.cc:487] T 480d5a3e251c4d1c983303a97ed07a41 P 96e3c2684cd4457685c687fcf04fce75 -> Peer 38213dd9fed845bf860fcb89a1fb8745 (127.7.86.129:41225): Couldn't send request to peer 38213dd9fed845bf860fcb89a1fb8745. Status: Network error: Client connection negotiation failed: client connection to 127.7.86.129:41225: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20250114 08:56:29.211606 7514 external_mini_cluster.cc:1620] Killing /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu with pid 7742
I20250114 08:56:29.234058 7514 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu pid 7872
W20250114 08:56:29.233981 7537 connection.cc:537] server connection from 127.7.86.130:33511 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250114 08:56:29.240620 7994 generic_service.cc:196] Checking for leaks (request via RPC)
I20250114 08:56:29.364367 7514 external_mini_cluster.cc:1620] Killing /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu with pid 7872
I20250114 08:56:29.388290 7514 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu pid 7522
I20250114 08:56:29.389447 7583 generic_service.cc:196] Checking for leaks (request via RPC)
I20250114 08:56:29.492611 7514 external_mini_cluster.cc:1620] Killing /tmp/dist-test-task4fHvvQ/build/asan/bin/kudu with pid 7522
2025-01-14T08:56:29Z chronyd exiting
I20250114 08:56:29.525876 7514 test_util.cc:181] -----------------------------------------------
I20250114 08:56:29.525959 7514 test_util.cc:182] Had failures, leaving test files at /tmp/dist-test-task4fHvvQ/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736844981707183-7514-0
[ FAILED ] ReplaceTabletITest.ReplaceTabletsWhileWriting (7752 ms)
[----------] 1 test from ReplaceTabletITest (7752 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (7752 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ReplaceTabletITest.ReplaceTabletsWhileWriting
1 FAILED TEST
I20250114 08:56:29.527262 7514 logging.cc:424] LogThrottler /home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/client/batcher.cc:441: suppressed but not reported on 3 messages since previous log ~2 seconds ago