[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from ReplaceTabletITest
[ RUN ] ReplaceTabletITest.ReplaceTabletsWhileWriting
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20250115 10:02:10.728924 7644 test_util.cc:274] Using random seed: -983381071
2025-01-15T10:02:10Z chronyd version 4.6.1 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -NTS -SECHASH -IPV6 +DEBUG)
2025-01-15T10:02:10Z Disabled control of system clock
I20250115 10:02:10.765585 7644 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu
/tmp/dist-test-taskeGzcyx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.119.62:33605
--webserver_interface=127.7.119.62
--webserver_port=0
--builtin_ntp_servers=127.7.119.20:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.119.62:33605 with env {}
W20250115 10:02:11.091109 7652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 10:02:11.091400 7652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 10:02:11.091491 7652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 10:02:11.099038 7652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20250115 10:02:11.099139 7652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 10:02:11.099189 7652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20250115 10:02:11.099246 7652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20250115 10:02:11.108103 7652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.119.20:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.119.62:33605
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.119.62:33605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.119.62
--webserver_port=0
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type FASTDEBUG
built by None at 15 Jan 2025 09:25:59 UTC on 24a791456cd2
build id 4187
ASAN enabled
I20250115 10:02:11.109606 7652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 10:02:11.111315 7652 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250115 10:02:11.121680 7657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 10:02:11.122864 7658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 10:02:11.123956 7660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 10:02:11.124951 7652 server_base.cc:1034] running on GCE node
I20250115 10:02:11.125653 7652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 10:02:11.126998 7652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 10:02:11.128212 7652 hybrid_clock.cc:648] HybridClock initialized: now 1736935331128145 us; error 38 us; skew 500 ppm
I20250115 10:02:11.128513 7652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250115 10:02:11.135790 7652 webserver.cc:458] Webserver started at http://127.7.119.62:44973/ using document root <none> and password file <none>
I20250115 10:02:11.136530 7652 fs_manager.cc:362] Metadata directory not provided
I20250115 10:02:11.136637 7652 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 10:02:11.136969 7652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 10:02:11.139658 7652 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/data/instance:
uuid: "91795ce3770449ca959a107f35c29a16"
format_stamp: "Formatted at 2025-01-15 10:02:11 on dist-test-slave-xhp2"
I20250115 10:02:11.140381 7652 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/wal/instance:
uuid: "91795ce3770449ca959a107f35c29a16"
format_stamp: "Formatted at 2025-01-15 10:02:11 on dist-test-slave-xhp2"
I20250115 10:02:11.145409 7652 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20250115 10:02:11.148419 7666 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 10:02:11.149673 7652 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20250115 10:02:11.149837 7652 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/data,/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/wal
uuid: "91795ce3770449ca959a107f35c29a16"
format_stamp: "Formatted at 2025-01-15 10:02:11 on dist-test-slave-xhp2"
I20250115 10:02:11.150007 7652 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 10:02:11.161790 7652 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 10:02:11.162508 7652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 10:02:11.162760 7652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 10:02:11.178771 7652 rpc_server.cc:307] RPC server started. Bound to: 127.7.119.62:33605
I20250115 10:02:11.178793 7717 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.119.62:33605 every 8 connection(s)
I20250115 10:02:11.180122 7652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/data/info.pb
I20250115 10:02:11.182596 7644 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu as pid 7652
I20250115 10:02:11.182915 7644 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/master-0/wal/instance
I20250115 10:02:11.184829 7718 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:11.195302 7718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16: Bootstrap starting.
I20250115 10:02:11.198555 7718 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:11.200095 7718 log.cc:826] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16: Log is configured to *not* fsync() on all Append() calls
I20250115 10:02:11.202994 7718 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16: No bootstrap required, opened a new log
I20250115 10:02:11.210949 7718 raft_consensus.cc:357] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91795ce3770449ca959a107f35c29a16" member_type: VOTER last_known_addr { host: "127.7.119.62" port: 33605 } }
I20250115 10:02:11.211395 7718 raft_consensus.cc:383] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:11.211519 7718 raft_consensus.cc:738] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91795ce3770449ca959a107f35c29a16, State: Initialized, Role: FOLLOWER
I20250115 10:02:11.212168 7718 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91795ce3770449ca959a107f35c29a16" member_type: VOTER last_known_addr { host: "127.7.119.62" port: 33605 } }
I20250115 10:02:11.212388 7718 raft_consensus.cc:397] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20250115 10:02:11.212510 7718 raft_consensus.cc:491] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20250115 10:02:11.212713 7718 raft_consensus.cc:3054] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:11.214744 7718 raft_consensus.cc:513] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91795ce3770449ca959a107f35c29a16" member_type: VOTER last_known_addr { host: "127.7.119.62" port: 33605 } }
I20250115 10:02:11.215202 7718 leader_election.cc:304] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91795ce3770449ca959a107f35c29a16; no voters:
I20250115 10:02:11.215741 7718 leader_election.cc:290] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20250115 10:02:11.215883 7723 raft_consensus.cc:2798] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 10:02:11.216470 7723 raft_consensus.cc:695] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [term 1 LEADER]: Becoming Leader. State: Replica: 91795ce3770449ca959a107f35c29a16, State: Running, Role: LEADER
I20250115 10:02:11.217049 7723 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91795ce3770449ca959a107f35c29a16" member_type: VOTER last_known_addr { host: "127.7.119.62" port: 33605 } }
I20250115 10:02:11.217700 7718 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [sys.catalog]: configured and running, proceeding with master startup.
I20250115 10:02:11.219820 7725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91795ce3770449ca959a107f35c29a16. Latest consensus state: current_term: 1 leader_uuid: "91795ce3770449ca959a107f35c29a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91795ce3770449ca959a107f35c29a16" member_type: VOTER last_known_addr { host: "127.7.119.62" port: 33605 } } }
I20250115 10:02:11.219852 7724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91795ce3770449ca959a107f35c29a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91795ce3770449ca959a107f35c29a16" member_type: VOTER last_known_addr { host: "127.7.119.62" port: 33605 } } }
I20250115 10:02:11.220239 7724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [sys.catalog]: This master's current role is: LEADER
I20250115 10:02:11.221072 7729 catalog_manager.cc:1476] Loading table and tablet metadata into memory...
I20250115 10:02:11.220109 7725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16 [sys.catalog]: This master's current role is: LEADER
I20250115 10:02:11.226469 7729 catalog_manager.cc:1485] Initializing Kudu cluster ID...
I20250115 10:02:11.235868 7729 catalog_manager.cc:1348] Generated new cluster ID: 0b1aca7e545640fd90e9f3f2aba196c1
I20250115 10:02:11.235977 7729 catalog_manager.cc:1496] Initializing Kudu internal certificate authority...
I20250115 10:02:11.256557 7729 catalog_manager.cc:1371] Generated new certificate authority record
I20250115 10:02:11.257755 7729 catalog_manager.cc:1505] Loading token signing keys...
I20250115 10:02:11.269433 7729 catalog_manager.cc:5899] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16: Generated new TSK 0
I20250115 10:02:11.270264 7729 catalog_manager.cc:1515] Initializing in-progress tserver states...
I20250115 10:02:11.289222 7644 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu
/tmp/dist-test-taskeGzcyx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.119.1:0
--local_ip_for_outbound_sockets=127.7.119.1
--webserver_interface=127.7.119.1
--webserver_port=0
--tserver_master_addrs=127.7.119.62:33605
--builtin_ntp_servers=127.7.119.20:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
W20250115 10:02:11.644497 7742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 10:02:11.644822 7742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 10:02:11.644924 7742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 10:02:11.652347 7742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 10:02:11.652525 7742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.119.1
I20250115 10:02:11.661597 7742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.119.20:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.119.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.119.1
--webserver_port=0
--tserver_master_addrs=127.7.119.62:33605
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.119.1
--log_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type FASTDEBUG
built by None at 15 Jan 2025 09:25:59 UTC on 24a791456cd2
build id 4187
ASAN enabled
I20250115 10:02:11.663075 7742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 10:02:11.664763 7742 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250115 10:02:11.676676 7747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 10:02:11.677949 7750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 10:02:11.678851 7748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 10:02:11.679870 7742 server_base.cc:1034] running on GCE node
I20250115 10:02:11.680442 7742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 10:02:11.681366 7742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 10:02:11.682566 7742 hybrid_clock.cc:648] HybridClock initialized: now 1736935331682496 us; error 49 us; skew 500 ppm
I20250115 10:02:11.682871 7742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250115 10:02:11.685082 7742 webserver.cc:458] Webserver started at http://127.7.119.1:44183/ using document root <none> and password file <none>
I20250115 10:02:11.685796 7742 fs_manager.cc:362] Metadata directory not provided
I20250115 10:02:11.685909 7742 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 10:02:11.686219 7742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 10:02:11.688902 7742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/data/instance:
uuid: "bf32bd013c6048c584442f708b0427a5"
format_stamp: "Formatted at 2025-01-15 10:02:11 on dist-test-slave-xhp2"
I20250115 10:02:11.689648 7742 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/wal/instance:
uuid: "bf32bd013c6048c584442f708b0427a5"
format_stamp: "Formatted at 2025-01-15 10:02:11 on dist-test-slave-xhp2"
I20250115 10:02:11.694643 7742 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20250115 10:02:11.697790 7756 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 10:02:11.699090 7742 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20250115 10:02:11.699244 7742 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/data,/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/wal
uuid: "bf32bd013c6048c584442f708b0427a5"
format_stamp: "Formatted at 2025-01-15 10:02:11 on dist-test-slave-xhp2"
I20250115 10:02:11.699415 7742 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 10:02:11.728873 7742 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 10:02:11.729702 7742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 10:02:11.729990 7742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 10:02:11.731009 7742 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250115 10:02:11.732463 7742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250115 10:02:11.732540 7742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250115 10:02:11.732697 7742 ts_tablet_manager.cc:610] Registered 0 tablets
I20250115 10:02:11.732760 7742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20250115 10:02:11.763255 7742 rpc_server.cc:307] RPC server started. Bound to: 127.7.119.1:38903
I20250115 10:02:11.763302 7868 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.119.1:38903 every 8 connection(s)
I20250115 10:02:11.764585 7742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/data/info.pb
I20250115 10:02:11.769620 7644 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu as pid 7742
I20250115 10:02:11.769814 7644 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/wal/instance
I20250115 10:02:11.773490 7644 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu
/tmp/dist-test-taskeGzcyx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.119.2:0
--local_ip_for_outbound_sockets=127.7.119.2
--webserver_interface=127.7.119.2
--webserver_port=0
--tserver_master_addrs=127.7.119.62:33605
--builtin_ntp_servers=127.7.119.20:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250115 10:02:11.778710 7869 heartbeater.cc:346] Connected to a master server at 127.7.119.62:33605
I20250115 10:02:11.779035 7869 heartbeater.cc:463] Registering TS with master...
I20250115 10:02:11.779814 7869 heartbeater.cc:509] Master 127.7.119.62:33605 requested a full tablet report, sending...
I20250115 10:02:11.782068 7683 ts_manager.cc:194] Registered new tserver with Master: bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903)
I20250115 10:02:11.784240 7683 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.119.1:40669
W20250115 10:02:12.095348 7873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 10:02:12.095652 7873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 10:02:12.095748 7873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 10:02:12.103302 7873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 10:02:12.103494 7873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.119.2
I20250115 10:02:12.112423 7873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.119.20:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.119.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.119.2
--webserver_port=0
--tserver_master_addrs=127.7.119.62:33605
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.119.2
--log_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type FASTDEBUG
built by None at 15 Jan 2025 09:25:59 UTC on 24a791456cd2
build id 4187
ASAN enabled
I20250115 10:02:12.113978 7873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 10:02:12.115589 7873 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250115 10:02:12.127352 7878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 10:02:12.128454 7881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 10:02:12.129287 7879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 10:02:12.130558 7873 server_base.cc:1034] running on GCE node
I20250115 10:02:12.131119 7873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 10:02:12.131985 7873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 10:02:12.133198 7873 hybrid_clock.cc:648] HybridClock initialized: now 1736935332133136 us; error 34 us; skew 500 ppm
I20250115 10:02:12.133507 7873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250115 10:02:12.135629 7873 webserver.cc:458] Webserver started at http://127.7.119.2:34489/ using document root <none> and password file <none>
I20250115 10:02:12.136351 7873 fs_manager.cc:362] Metadata directory not provided
I20250115 10:02:12.136463 7873 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 10:02:12.136785 7873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 10:02:12.139302 7873 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/data/instance:
uuid: "236c0d893ac742beb1c31a07f33ce84d"
format_stamp: "Formatted at 2025-01-15 10:02:12 on dist-test-slave-xhp2"
I20250115 10:02:12.140049 7873 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/wal/instance:
uuid: "236c0d893ac742beb1c31a07f33ce84d"
format_stamp: "Formatted at 2025-01-15 10:02:12 on dist-test-slave-xhp2"
I20250115 10:02:12.145042 7873 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20250115 10:02:12.148047 7887 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 10:02:12.149267 7873 fs_manager.cc:730] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20250115 10:02:12.149416 7873 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/data,/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/wal
uuid: "236c0d893ac742beb1c31a07f33ce84d"
format_stamp: "Formatted at 2025-01-15 10:02:12 on dist-test-slave-xhp2"
I20250115 10:02:12.149574 7873 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 10:02:12.163558 7873 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 10:02:12.164324 7873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 10:02:12.164592 7873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 10:02:12.165542 7873 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250115 10:02:12.166977 7873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250115 10:02:12.167047 7873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250115 10:02:12.167133 7873 ts_tablet_manager.cc:610] Registered 0 tablets
I20250115 10:02:12.167176 7873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250115 10:02:12.198534 7873 rpc_server.cc:307] RPC server started. Bound to: 127.7.119.2:39047
I20250115 10:02:12.198619 7999 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.119.2:39047 every 8 connection(s)
I20250115 10:02:12.199841 7873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/data/info.pb
I20250115 10:02:12.203254 7644 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu as pid 7873
I20250115 10:02:12.203485 7644 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/wal/instance
I20250115 10:02:12.207203 7644 external_mini_cluster.cc:1351] Running /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu
/tmp/dist-test-taskeGzcyx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.119.3:0
--local_ip_for_outbound_sockets=127.7.119.3
--webserver_interface=127.7.119.3
--webserver_port=0
--tserver_master_addrs=127.7.119.62:33605
--builtin_ntp_servers=127.7.119.20:42269
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin with env {}
I20250115 10:02:12.213800 8000 heartbeater.cc:346] Connected to a master server at 127.7.119.62:33605
I20250115 10:02:12.214147 8000 heartbeater.cc:463] Registering TS with master...
I20250115 10:02:12.215204 8000 heartbeater.cc:509] Master 127.7.119.62:33605 requested a full tablet report, sending...
I20250115 10:02:12.216832 7683 ts_manager.cc:194] Registered new tserver with Master: 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047)
I20250115 10:02:12.217612 7683 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.119.2:37769
W20250115 10:02:12.539826 8003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20250115 10:02:12.540105 8003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20250115 10:02:12.540259 8003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20250115 10:02:12.548027 8003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20250115 10:02:12.548256 8003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.119.3
I20250115 10:02:12.557377 8003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.119.20:42269
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/wal
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.119.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.119.3
--webserver_port=0
--tserver_master_addrs=127.7.119.62:33605
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.119.3
--log_dir=/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision 75b75ca2f58f1a5aba69207830e3558011033d99
build type FASTDEBUG
built by None at 15 Jan 2025 09:25:59 UTC on 24a791456cd2
build id 4187
ASAN enabled
I20250115 10:02:12.558918 8003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20250115 10:02:12.560601 8003 file_cache.cc:492] Constructed file cache file cache with capacity 419430
W20250115 10:02:12.572679 8008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 10:02:12.573920 8009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20250115 10:02:12.574855 8011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20250115 10:02:12.575745 8003 server_base.cc:1034] running on GCE node
I20250115 10:02:12.576385 8003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20250115 10:02:12.577323 8003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20250115 10:02:12.578533 8003 hybrid_clock.cc:648] HybridClock initialized: now 1736935332578467 us; error 37 us; skew 500 ppm
I20250115 10:02:12.578858 8003 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20250115 10:02:12.581089 8003 webserver.cc:458] Webserver started at http://127.7.119.3:46671/ using document root <none> and password file <none>
I20250115 10:02:12.581810 8003 fs_manager.cc:362] Metadata directory not provided
I20250115 10:02:12.581925 8003 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20250115 10:02:12.582221 8003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20250115 10:02:12.584853 8003 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/data/instance:
uuid: "ad9d5d231fad4f0290f5d987cea32078"
format_stamp: "Formatted at 2025-01-15 10:02:12 on dist-test-slave-xhp2"
I20250115 10:02:12.585618 8003 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/wal/instance:
uuid: "ad9d5d231fad4f0290f5d987cea32078"
format_stamp: "Formatted at 2025-01-15 10:02:12 on dist-test-slave-xhp2"
I20250115 10:02:12.590807 8003 fs_manager.cc:696] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20250115 10:02:12.594075 8017 log_block_manager.cc:3788] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20250115 10:02:12.595314 8003 fs_manager.cc:730] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20250115 10:02:12.595472 8003 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/data,/tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/wal
uuid: "ad9d5d231fad4f0290f5d987cea32078"
format_stamp: "Formatted at 2025-01-15 10:02:12 on dist-test-slave-xhp2"
I20250115 10:02:12.595645 8003 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20250115 10:02:12.613032 8003 rpc_server.cc:225] running with OpenSSL 1.1.1 11 Sep 2018
I20250115 10:02:12.613790 8003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20250115 10:02:12.614058 8003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20250115 10:02:12.614928 8003 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20250115 10:02:12.616335 8003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20250115 10:02:12.616415 8003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20250115 10:02:12.616544 8003 ts_tablet_manager.cc:610] Registered 0 tablets
I20250115 10:02:12.616601 8003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20250115 10:02:12.647864 8003 rpc_server.cc:307] RPC server started. Bound to: 127.7.119.3:34549
I20250115 10:02:12.647919 8129 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.7.119.3:34549 every 8 connection(s)
I20250115 10:02:12.649216 8003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/data/info.pb
I20250115 10:02:12.657657 7644 external_mini_cluster.cc:1413] Started /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu as pid 8003
I20250115 10:02:12.657814 7644 external_mini_cluster.cc:1427] Reading /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/wal/instance
I20250115 10:02:12.661895 8130 heartbeater.cc:346] Connected to a master server at 127.7.119.62:33605
I20250115 10:02:12.662168 8130 heartbeater.cc:463] Registering TS with master...
I20250115 10:02:12.662909 8130 heartbeater.cc:509] Master 127.7.119.62:33605 requested a full tablet report, sending...
I20250115 10:02:12.664165 7683 ts_manager.cc:194] Registered new tserver with Master: ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3:34549)
I20250115 10:02:12.664984 7683 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.119.3:55035
I20250115 10:02:12.674032 7644 external_mini_cluster.cc:934] 3 TS(s) registered with all masters
I20250115 10:02:12.702023 7644 test_util.cc:274] Using random seed: -981407955
I20250115 10:02:12.722198 7683 catalog_manager.cc:1909] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34550:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\004\001\000\377\377\377\037\004\001\000\376\377\377?\004\001\000\375\377\377_"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20250115 10:02:12.724076 7683 catalog_manager.cc:6885] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20250115 10:02:12.758652 7935 tablet_service.cc:1467] Processing CreateTablet for tablet b858517ace534efc9be532652cf101a1 (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250115 10:02:12.758653 8064 tablet_service.cc:1467] Processing CreateTablet for tablet 0ff0cd8db6b146e3a590c02ce7dac65e (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250115 10:02:12.759049 8063 tablet_service.cc:1467] Processing CreateTablet for tablet 376139a85f9140ffb1d60b2a305ac60a (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250115 10:02:12.760365 7932 tablet_service.cc:1467] Processing CreateTablet for tablet 88ef3f8473424a3aad263db0aacb6148 (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250115 10:02:12.760883 8064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff0cd8db6b146e3a590c02ce7dac65e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.759789 7934 tablet_service.cc:1467] Processing CreateTablet for tablet 0ff0cd8db6b146e3a590c02ce7dac65e (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250115 10:02:12.759202 7933 tablet_service.cc:1467] Processing CreateTablet for tablet 376139a85f9140ffb1d60b2a305ac60a (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250115 10:02:12.761032 8062 tablet_service.cc:1467] Processing CreateTablet for tablet 88ef3f8473424a3aad263db0aacb6148 (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250115 10:02:12.760890 8065 tablet_service.cc:1467] Processing CreateTablet for tablet b858517ace534efc9be532652cf101a1 (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250115 10:02:12.761459 7935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b858517ace534efc9be532652cf101a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.761502 8062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ef3f8473424a3aad263db0aacb6148. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.761850 8065 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b858517ace534efc9be532652cf101a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.762037 7933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 376139a85f9140ffb1d60b2a305ac60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.762537 7934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff0cd8db6b146e3a590c02ce7dac65e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.763474 8063 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 376139a85f9140ffb1d60b2a305ac60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.766050 7932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ef3f8473424a3aad263db0aacb6148. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.785429 8158 tablet_bootstrap.cc:492] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: Bootstrap starting.
I20250115 10:02:12.786185 7804 tablet_service.cc:1467] Processing CreateTablet for tablet b858517ace534efc9be532652cf101a1 (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION VALUES < 536870911
I20250115 10:02:12.785181 7802 tablet_service.cc:1467] Processing CreateTablet for tablet 376139a85f9140ffb1d60b2a305ac60a (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 1073741822 <= VALUES < 1610612733
I20250115 10:02:12.785143 7803 tablet_service.cc:1467] Processing CreateTablet for tablet 0ff0cd8db6b146e3a590c02ce7dac65e (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250115 10:02:12.787415 7804 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b858517ace534efc9be532652cf101a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.787896 7803 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ff0cd8db6b146e3a590c02ce7dac65e. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.788915 7869 heartbeater.cc:501] Master 127.7.119.62:33605 was elected leader, sending a full tablet report...
I20250115 10:02:12.790252 8158 tablet_bootstrap.cc:654] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.785915 7801 tablet_service.cc:1467] Processing CreateTablet for tablet 88ef3f8473424a3aad263db0aacb6148 (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 1610612733 <= VALUES
I20250115 10:02:12.790763 7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 88ef3f8473424a3aad263db0aacb6148. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.791919 7802 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 376139a85f9140ffb1d60b2a305ac60a. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:12.791893 8158 log.cc:826] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: Log is configured to *not* fsync() on all Append() calls
I20250115 10:02:12.800619 8159 tablet_bootstrap.cc:492] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078: Bootstrap starting.
I20250115 10:02:12.803265 8158 tablet_bootstrap.cc:492] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: No bootstrap required, opened a new log
I20250115 10:02:12.803582 8158 ts_tablet_manager.cc:1397] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: Time spent bootstrapping tablet: real 0.018s user 0.006s sys 0.007s
I20250115 10:02:12.805886 8159 tablet_bootstrap.cc:654] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.807340 8159 log.cc:826] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078: Log is configured to *not* fsync() on all Append() calls
I20250115 10:02:12.814842 8158 raft_consensus.cc:357] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.815510 8158 raft_consensus.cc:383] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.815670 8158 raft_consensus.cc:738] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236c0d893ac742beb1c31a07f33ce84d, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.816115 8162 tablet_bootstrap.cc:492] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5: Bootstrap starting.
I20250115 10:02:12.816501 8158 consensus_queue.cc:260] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.817565 8159 tablet_bootstrap.cc:492] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078: No bootstrap required, opened a new log
I20250115 10:02:12.817766 8000 heartbeater.cc:501] Master 127.7.119.62:33605 was elected leader, sending a full tablet report...
I20250115 10:02:12.817880 8159 ts_tablet_manager.cc:1397] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078: Time spent bootstrapping tablet: real 0.018s user 0.010s sys 0.004s
I20250115 10:02:12.818375 8158 ts_tablet_manager.cc:1428] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20250115 10:02:12.818955 8158 tablet_bootstrap.cc:492] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d: Bootstrap starting.
I20250115 10:02:12.819752 8162 tablet_bootstrap.cc:654] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.820816 8162 log.cc:826] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5: Log is configured to *not* fsync() on all Append() calls
I20250115 10:02:12.824844 8158 tablet_bootstrap.cc:654] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.827175 8159 raft_consensus.cc:357] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.827582 8159 raft_consensus.cc:383] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.827692 8159 raft_consensus.cc:738] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9d5d231fad4f0290f5d987cea32078, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.828461 8158 tablet_bootstrap.cc:492] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d: No bootstrap required, opened a new log
I20250115 10:02:12.828485 8159 consensus_queue.cc:260] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.828701 8158 ts_tablet_manager.cc:1397] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.004s
I20250115 10:02:12.829792 8130 heartbeater.cc:501] Master 127.7.119.62:33605 was elected leader, sending a full tablet report...
I20250115 10:02:12.829814 8158 raft_consensus.cc:357] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.830070 8158 raft_consensus.cc:383] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.830219 8158 raft_consensus.cc:738] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236c0d893ac742beb1c31a07f33ce84d, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.830613 8159 ts_tablet_manager.cc:1428] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20250115 10:02:12.830590 8158 consensus_queue.cc:260] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.831099 8159 tablet_bootstrap.cc:492] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078: Bootstrap starting.
I20250115 10:02:12.831421 8162 tablet_bootstrap.cc:492] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5: No bootstrap required, opened a new log
I20250115 10:02:12.831735 8162 ts_tablet_manager.cc:1397] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20250115 10:02:12.833516 8158 ts_tablet_manager.cc:1428] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20250115 10:02:12.833823 8158 tablet_bootstrap.cc:492] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d: Bootstrap starting.
I20250115 10:02:12.834120 8159 tablet_bootstrap.cc:654] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.835639 8158 tablet_bootstrap.cc:654] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.842444 8159 tablet_bootstrap.cc:492] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078: No bootstrap required, opened a new log
I20250115 10:02:12.842576 8159 ts_tablet_manager.cc:1397] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.003s
I20250115 10:02:12.842654 8162 raft_consensus.cc:357] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.843093 8162 raft_consensus.cc:383] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.843202 8162 raft_consensus.cc:738] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf32bd013c6048c584442f708b0427a5, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.843389 8158 tablet_bootstrap.cc:492] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d: No bootstrap required, opened a new log
I20250115 10:02:12.843566 8158 ts_tablet_manager.cc:1397] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.005s
I20250115 10:02:12.843596 8159 raft_consensus.cc:357] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.843824 8159 raft_consensus.cc:383] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.843917 8159 raft_consensus.cc:738] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9d5d231fad4f0290f5d987cea32078, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.843936 8162 consensus_queue.cc:260] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.844147 8159 consensus_queue.cc:260] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.844383 8158 raft_consensus.cc:357] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.844623 8158 raft_consensus.cc:383] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.844736 8159 ts_tablet_manager.cc:1428] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250115 10:02:12.844758 8158 raft_consensus.cc:738] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236c0d893ac742beb1c31a07f33ce84d, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.845062 8158 consensus_queue.cc:260] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.845443 8159 tablet_bootstrap.cc:492] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: Bootstrap starting.
I20250115 10:02:12.846580 8158 ts_tablet_manager.cc:1428] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20250115 10:02:12.846962 8158 tablet_bootstrap.cc:492] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d: Bootstrap starting.
I20250115 10:02:12.847771 8159 tablet_bootstrap.cc:654] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.849875 8158 tablet_bootstrap.cc:654] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.852038 8158 tablet_bootstrap.cc:492] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d: No bootstrap required, opened a new log
I20250115 10:02:12.852180 8158 ts_tablet_manager.cc:1397] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20250115 10:02:12.852818 8158 raft_consensus.cc:357] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.852890 8162 ts_tablet_manager.cc:1428] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5: Time spent starting tablet: real 0.021s user 0.015s sys 0.005s
I20250115 10:02:12.852982 8158 raft_consensus.cc:383] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.853084 8158 raft_consensus.cc:738] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236c0d893ac742beb1c31a07f33ce84d, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.853317 8162 tablet_bootstrap.cc:492] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5: Bootstrap starting.
I20250115 10:02:12.853305 8158 consensus_queue.cc:260] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.853888 8158 ts_tablet_manager.cc:1428] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250115 10:02:12.855018 8162 tablet_bootstrap.cc:654] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.855273 8159 tablet_bootstrap.cc:492] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: No bootstrap required, opened a new log
I20250115 10:02:12.855481 8159 ts_tablet_manager.cc:1397] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20250115 10:02:12.856417 8159 raft_consensus.cc:357] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.856627 8159 raft_consensus.cc:383] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.856736 8159 raft_consensus.cc:738] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9d5d231fad4f0290f5d987cea32078, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.856997 8159 consensus_queue.cc:260] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.857578 8162 tablet_bootstrap.cc:492] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5: No bootstrap required, opened a new log
I20250115 10:02:12.857681 8159 ts_tablet_manager.cc:1428] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20250115 10:02:12.857738 8162 ts_tablet_manager.cc:1397] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20250115 10:02:12.858001 8159 tablet_bootstrap.cc:492] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078: Bootstrap starting.
I20250115 10:02:12.858702 8162 raft_consensus.cc:357] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.858983 8162 raft_consensus.cc:383] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.859113 8162 raft_consensus.cc:738] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf32bd013c6048c584442f708b0427a5, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.859364 8162 consensus_queue.cc:260] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.859747 8159 tablet_bootstrap.cc:654] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.859949 8162 ts_tablet_manager.cc:1428] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20250115 10:02:12.860229 8162 tablet_bootstrap.cc:492] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: Bootstrap starting.
I20250115 10:02:12.862457 8162 tablet_bootstrap.cc:654] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.863435 8159 tablet_bootstrap.cc:492] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078: No bootstrap required, opened a new log
I20250115 10:02:12.863586 8159 ts_tablet_manager.cc:1397] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20250115 10:02:12.864341 8159 raft_consensus.cc:357] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.864473 8162 tablet_bootstrap.cc:492] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: No bootstrap required, opened a new log
I20250115 10:02:12.864554 8159 raft_consensus.cc:383] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.864636 8162 ts_tablet_manager.cc:1397] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250115 10:02:12.864677 8159 raft_consensus.cc:738] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9d5d231fad4f0290f5d987cea32078, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.864913 8159 consensus_queue.cc:260] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.865482 8159 ts_tablet_manager.cc:1428] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20250115 10:02:12.865428 8162 raft_consensus.cc:357] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.865693 8162 raft_consensus.cc:383] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.865779 8162 raft_consensus.cc:738] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf32bd013c6048c584442f708b0427a5, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.866001 8162 consensus_queue.cc:260] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.866407 8162 ts_tablet_manager.cc:1428] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20250115 10:02:12.866667 8162 tablet_bootstrap.cc:492] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5: Bootstrap starting.
I20250115 10:02:12.868280 8162 tablet_bootstrap.cc:654] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:12.870110 8162 tablet_bootstrap.cc:492] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5: No bootstrap required, opened a new log
I20250115 10:02:12.870254 8162 ts_tablet_manager.cc:1397] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20250115 10:02:12.870821 8162 raft_consensus.cc:357] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.870962 8162 raft_consensus.cc:383] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:12.871057 8162 raft_consensus.cc:738] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf32bd013c6048c584442f708b0427a5, State: Initialized, Role: FOLLOWER
I20250115 10:02:12.871240 8162 consensus_queue.cc:260] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.871652 8162 ts_tablet_manager.cc:1428] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20250115 10:02:12.900890 8131 tablet.cc:2367] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250115 10:02:12.949236 8169 raft_consensus.cc:491] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 10:02:12.949569 8169 raft_consensus.cc:513] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.950687 8166 raft_consensus.cc:491] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 10:02:12.950927 8166 raft_consensus.cc:513] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
W20250115 10:02:12.951884 8001 tablet.cc:2367] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250115 10:02:12.952989 8166 leader_election.cc:290] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047), bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903)
I20250115 10:02:12.953655 8169 leader_election.cc:290] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3:34549), 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047)
I20250115 10:02:12.962229 7824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858517ace534efc9be532652cf101a1" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf32bd013c6048c584442f708b0427a5" is_pre_election: true
I20250115 10:02:12.962685 7824 raft_consensus.cc:2463] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9d5d231fad4f0290f5d987cea32078 in term 0.
I20250115 10:02:12.962821 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858517ace534efc9be532652cf101a1" candidate_uuid: "bf32bd013c6048c584442f708b0427a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d" is_pre_election: true
I20250115 10:02:12.963527 7955 raft_consensus.cc:2463] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf32bd013c6048c584442f708b0427a5 in term 0.
I20250115 10:02:12.964655 8019 leader_election.cc:304] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad9d5d231fad4f0290f5d987cea32078, bf32bd013c6048c584442f708b0427a5; no voters:
I20250115 10:02:12.965339 8166 raft_consensus.cc:2798] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 10:02:12.965530 8166 raft_consensus.cc:491] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 10:02:12.965660 8166 raft_consensus.cc:3054] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:12.966724 7760 leader_election.cc:304] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236c0d893ac742beb1c31a07f33ce84d, bf32bd013c6048c584442f708b0427a5; no voters:
I20250115 10:02:12.967186 8169 raft_consensus.cc:2798] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 10:02:12.967353 8169 raft_consensus.cc:491] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 10:02:12.967459 8169 raft_consensus.cc:3054] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:12.969504 8169 raft_consensus.cc:513] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.969594 8166 raft_consensus.cc:513] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.970095 8169 leader_election.cc:290] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [CANDIDATE]: Term 1 election: Requested vote from peers ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3:34549), 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047)
I20250115 10:02:12.970753 8163 raft_consensus.cc:491] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 10:02:12.970728 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858517ace534efc9be532652cf101a1" candidate_uuid: "bf32bd013c6048c584442f708b0427a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d"
I20250115 10:02:12.970891 7824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858517ace534efc9be532652cf101a1" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf32bd013c6048c584442f708b0427a5"
I20250115 10:02:12.970964 7955 raft_consensus.cc:3054] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:12.971175 7824 raft_consensus.cc:2388] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ad9d5d231fad4f0290f5d987cea32078 in current term 1: Already voted for candidate bf32bd013c6048c584442f708b0427a5 in this term.
I20250115 10:02:12.970927 8163 raft_consensus.cc:513] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.970357 8166 leader_election.cc:290] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 election: Requested vote from peers 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047), bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903)
I20250115 10:02:12.972932 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858517ace534efc9be532652cf101a1" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d" is_pre_election: true
I20250115 10:02:12.973068 7953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858517ace534efc9be532652cf101a1" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d"
I20250115 10:02:12.973619 8163 leader_election.cc:290] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903), ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3:34549)
I20250115 10:02:12.974509 8085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858517ace534efc9be532652cf101a1" candidate_uuid: "bf32bd013c6048c584442f708b0427a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9d5d231fad4f0290f5d987cea32078" is_pre_election: true
I20250115 10:02:12.974545 8084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b858517ace534efc9be532652cf101a1" candidate_uuid: "bf32bd013c6048c584442f708b0427a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9d5d231fad4f0290f5d987cea32078"
I20250115 10:02:12.974911 8084 raft_consensus.cc:2388] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bf32bd013c6048c584442f708b0427a5 in current term 1: Already voted for candidate ad9d5d231fad4f0290f5d987cea32078 in this term.
I20250115 10:02:12.975886 7955 raft_consensus.cc:2463] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf32bd013c6048c584442f708b0427a5 in term 1.
I20250115 10:02:12.976262 7760 leader_election.cc:304] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 236c0d893ac742beb1c31a07f33ce84d, bf32bd013c6048c584442f708b0427a5; no voters: ad9d5d231fad4f0290f5d987cea32078
I20250115 10:02:12.976634 8169 raft_consensus.cc:2798] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 10:02:12.977329 8021 leader_election.cc:304] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad9d5d231fad4f0290f5d987cea32078; no voters: 236c0d893ac742beb1c31a07f33ce84d, bf32bd013c6048c584442f708b0427a5
I20250115 10:02:12.977418 8169 raft_consensus.cc:695] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [term 1 LEADER]: Becoming Leader. State: Replica: bf32bd013c6048c584442f708b0427a5, State: Running, Role: LEADER
I20250115 10:02:12.977658 8166 raft_consensus.cc:2743] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20250115 10:02:12.977934 8169 consensus_queue.cc:237] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } }
I20250115 10:02:12.981552 8166 raft_consensus.cc:491] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 10:02:12.981722 8166 raft_consensus.cc:513] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.982388 8166 leader_election.cc:290] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903), 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047)
I20250115 10:02:12.983003 7824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376139a85f9140ffb1d60b2a305ac60a" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf32bd013c6048c584442f708b0427a5" is_pre_election: true
I20250115 10:02:12.983191 7824 raft_consensus.cc:2463] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9d5d231fad4f0290f5d987cea32078 in term 0.
I20250115 10:02:12.983654 8019 leader_election.cc:304] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad9d5d231fad4f0290f5d987cea32078, bf32bd013c6048c584442f708b0427a5; no voters:
I20250115 10:02:12.983492 7683 catalog_manager.cc:5526] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf32bd013c6048c584442f708b0427a5 (127.7.119.1). New cstate: current_term: 1 leader_uuid: "bf32bd013c6048c584442f708b0427a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } health_report { overall_health: HEALTHY } } }
I20250115 10:02:12.984025 8166 raft_consensus.cc:2798] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 10:02:12.984019 8084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ef3f8473424a3aad263db0aacb6148" candidate_uuid: "236c0d893ac742beb1c31a07f33ce84d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9d5d231fad4f0290f5d987cea32078" is_pre_election: true
I20250115 10:02:12.984138 8166 raft_consensus.cc:491] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 10:02:12.984244 8166 raft_consensus.cc:3054] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:12.984251 8084 raft_consensus.cc:2463] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 236c0d893ac742beb1c31a07f33ce84d in term 0.
I20250115 10:02:12.985951 7889 leader_election.cc:304] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236c0d893ac742beb1c31a07f33ce84d, ad9d5d231fad4f0290f5d987cea32078; no voters:
I20250115 10:02:12.986361 8163 raft_consensus.cc:2798] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 10:02:12.986513 8163 raft_consensus.cc:491] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 10:02:12.986619 8163 raft_consensus.cc:3054] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:12.986593 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376139a85f9140ffb1d60b2a305ac60a" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d" is_pre_election: true
I20250115 10:02:12.986826 7955 raft_consensus.cc:2463] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9d5d231fad4f0290f5d987cea32078 in term 0.
I20250115 10:02:12.986774 8166 raft_consensus.cc:513] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.987787 8166 leader_election.cc:290] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 election: Requested vote from peers bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903), 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047)
I20250115 10:02:12.987728 7824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376139a85f9140ffb1d60b2a305ac60a" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf32bd013c6048c584442f708b0427a5"
I20250115 10:02:12.987926 7824 raft_consensus.cc:3054] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:12.988425 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "376139a85f9140ffb1d60b2a305ac60a" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d"
I20250115 10:02:12.988461 7823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ef3f8473424a3aad263db0aacb6148" candidate_uuid: "236c0d893ac742beb1c31a07f33ce84d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf32bd013c6048c584442f708b0427a5" is_pre_election: true
I20250115 10:02:12.988626 7955 raft_consensus.cc:3054] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:12.988785 7823 raft_consensus.cc:2463] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 236c0d893ac742beb1c31a07f33ce84d in term 0.
I20250115 10:02:12.989492 8163 raft_consensus.cc:513] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.990507 8163 leader_election.cc:290] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [CANDIDATE]: Term 1 election: Requested vote from peers bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903), ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3:34549)
I20250115 10:02:12.990942 8084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ef3f8473424a3aad263db0aacb6148" candidate_uuid: "236c0d893ac742beb1c31a07f33ce84d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9d5d231fad4f0290f5d987cea32078"
I20250115 10:02:12.990942 7823 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88ef3f8473424a3aad263db0aacb6148" candidate_uuid: "236c0d893ac742beb1c31a07f33ce84d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf32bd013c6048c584442f708b0427a5"
I20250115 10:02:12.991122 7823 raft_consensus.cc:3054] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:12.991117 8084 raft_consensus.cc:3054] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:12.991304 7824 raft_consensus.cc:2463] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9d5d231fad4f0290f5d987cea32078 in term 1.
I20250115 10:02:12.991389 7955 raft_consensus.cc:2463] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9d5d231fad4f0290f5d987cea32078 in term 1.
I20250115 10:02:12.991724 8019 leader_election.cc:304] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad9d5d231fad4f0290f5d987cea32078, bf32bd013c6048c584442f708b0427a5; no voters:
I20250115 10:02:12.991993 8166 raft_consensus.cc:2798] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 10:02:12.992561 8166 raft_consensus.cc:695] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [term 1 LEADER]: Becoming Leader. State: Replica: ad9d5d231fad4f0290f5d987cea32078, State: Running, Role: LEADER
I20250115 10:02:12.993283 8166 consensus_queue.cc:237] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.993902 7823 raft_consensus.cc:2463] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 236c0d893ac742beb1c31a07f33ce84d in term 1.
I20250115 10:02:12.994256 7889 leader_election.cc:304] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236c0d893ac742beb1c31a07f33ce84d, bf32bd013c6048c584442f708b0427a5; no voters:
I20250115 10:02:12.994306 8084 raft_consensus.cc:2463] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 236c0d893ac742beb1c31a07f33ce84d in term 1.
I20250115 10:02:12.994549 8163 raft_consensus.cc:2798] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Leader election won for term 1
I20250115 10:02:12.994995 8163 raft_consensus.cc:695] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [term 1 LEADER]: Becoming Leader. State: Replica: 236c0d893ac742beb1c31a07f33ce84d, State: Running, Role: LEADER
I20250115 10:02:12.995718 8163 consensus_queue.cc:237] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:12.999547 7682 catalog_manager.cc:5526] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d reported cstate change: term changed from 0 to 1, leader changed from <none> to 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2). New cstate: current_term: 1 leader_uuid: "236c0d893ac742beb1c31a07f33ce84d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } health_report { overall_health: UNKNOWN } } }
I20250115 10:02:12.998637 7683 catalog_manager.cc:5526] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3). New cstate: current_term: 1 leader_uuid: "ad9d5d231fad4f0290f5d987cea32078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } health_report { overall_health: HEALTHY } } }
W20250115 10:02:13.017067 7870 tablet.cc:2367] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20250115 10:02:13.122351 8169 raft_consensus.cc:491] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 10:02:13.122591 8169 raft_consensus.cc:513] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:13.123250 8169 leader_election.cc:290] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047), ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3:34549)
I20250115 10:02:13.123932 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff0cd8db6b146e3a590c02ce7dac65e" candidate_uuid: "bf32bd013c6048c584442f708b0427a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d" is_pre_election: true
I20250115 10:02:13.124199 7955 raft_consensus.cc:2463] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf32bd013c6048c584442f708b0427a5 in term 0.
I20250115 10:02:13.124222 8084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff0cd8db6b146e3a590c02ce7dac65e" candidate_uuid: "bf32bd013c6048c584442f708b0427a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9d5d231fad4f0290f5d987cea32078" is_pre_election: true
I20250115 10:02:13.124495 8084 raft_consensus.cc:2463] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf32bd013c6048c584442f708b0427a5 in term 0.
I20250115 10:02:13.124684 7760 leader_election.cc:304] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236c0d893ac742beb1c31a07f33ce84d, bf32bd013c6048c584442f708b0427a5; no voters:
I20250115 10:02:13.124981 8169 raft_consensus.cc:2798] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 10:02:13.125082 8169 raft_consensus.cc:491] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 10:02:13.125186 8169 raft_consensus.cc:3054] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:13.126986 8169 raft_consensus.cc:513] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:13.127513 8169 leader_election.cc:290] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [CANDIDATE]: Term 1 election: Requested vote from peers 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047), ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3:34549)
I20250115 10:02:13.127907 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff0cd8db6b146e3a590c02ce7dac65e" candidate_uuid: "bf32bd013c6048c584442f708b0427a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d"
I20250115 10:02:13.127921 8084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0ff0cd8db6b146e3a590c02ce7dac65e" candidate_uuid: "bf32bd013c6048c584442f708b0427a5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad9d5d231fad4f0290f5d987cea32078"
I20250115 10:02:13.128072 7955 raft_consensus.cc:3054] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:13.128106 8084 raft_consensus.cc:3054] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:13.129951 7955 raft_consensus.cc:2463] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf32bd013c6048c584442f708b0427a5 in term 1.
I20250115 10:02:13.130023 8084 raft_consensus.cc:2463] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf32bd013c6048c584442f708b0427a5 in term 1.
I20250115 10:02:13.130311 7760 leader_election.cc:304] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 236c0d893ac742beb1c31a07f33ce84d, bf32bd013c6048c584442f708b0427a5; no voters:
I20250115 10:02:13.130628 8169 raft_consensus.cc:2798] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 10:02:13.130777 8169 raft_consensus.cc:695] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 1 LEADER]: Becoming Leader. State: Replica: bf32bd013c6048c584442f708b0427a5, State: Running, Role: LEADER
I20250115 10:02:13.131039 8169 consensus_queue.cc:237] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } }
I20250115 10:02:13.133682 7683 catalog_manager.cc:5526] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 reported cstate change: term changed from 0 to 1, leader changed from <none> to bf32bd013c6048c584442f708b0427a5 (127.7.119.1). New cstate: current_term: 1 leader_uuid: "bf32bd013c6048c584442f708b0427a5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } health_report { overall_health: UNKNOWN } } }
I20250115 10:02:13.265591 7955 raft_consensus.cc:1270] T 376139a85f9140ffb1d60b2a305ac60a P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Refusing update from remote peer ad9d5d231fad4f0290f5d987cea32078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 10:02:13.266578 8184 consensus_queue.cc:1035] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 10:02:13.266786 7823 raft_consensus.cc:1270] T 376139a85f9140ffb1d60b2a305ac60a P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Refusing update from remote peer ad9d5d231fad4f0290f5d987cea32078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 10:02:13.267681 8184 consensus_queue.cc:1035] T 376139a85f9140ffb1d60b2a305ac60a P ad9d5d231fad4f0290f5d987cea32078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 10:02:13.270916 8084 raft_consensus.cc:1270] T b858517ace534efc9be532652cf101a1 P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Refusing update from remote peer bf32bd013c6048c584442f708b0427a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 10:02:13.271740 8182 consensus_queue.cc:1035] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 10:02:13.273761 7954 raft_consensus.cc:1270] T b858517ace534efc9be532652cf101a1 P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Refusing update from remote peer bf32bd013c6048c584442f708b0427a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 10:02:13.274165 7953 raft_consensus.cc:1270] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Refusing update from remote peer bf32bd013c6048c584442f708b0427a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 10:02:13.274343 8182 consensus_queue.cc:1035] T b858517ace534efc9be532652cf101a1 P bf32bd013c6048c584442f708b0427a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 10:02:13.275498 8182 consensus_queue.cc:1035] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 10:02:13.276342 8084 raft_consensus.cc:1270] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Refusing update from remote peer bf32bd013c6048c584442f708b0427a5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 10:02:13.277096 8182 consensus_queue.cc:1035] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 10:02:13.305558 8199 mvcc.cc:204] Tried to move back new op lower bound from 7114487125068701696 to 7114487124506460160. Current Snapshot: MvccSnapshot[applied={T|T < 7114487125068701696}]
I20250115 10:02:13.309592 8198 mvcc.cc:204] Tried to move back new op lower bound from 7114487125068701696 to 7114487124506460160. Current Snapshot: MvccSnapshot[applied={T|T < 7114487125068701696}]
I20250115 10:02:13.314710 8084 raft_consensus.cc:1270] T 88ef3f8473424a3aad263db0aacb6148 P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Refusing update from remote peer 236c0d893ac742beb1c31a07f33ce84d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 10:02:13.315528 8163 consensus_queue.cc:1035] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 10:02:13.326665 7824 raft_consensus.cc:1270] T 88ef3f8473424a3aad263db0aacb6148 P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Refusing update from remote peer 236c0d893ac742beb1c31a07f33ce84d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20250115 10:02:13.327348 8163 consensus_queue.cc:1035] T 88ef3f8473424a3aad263db0aacb6148 P 236c0d893ac742beb1c31a07f33ce84d [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20250115 10:02:13.370126 8207 mvcc.cc:204] Tried to move back new op lower bound from 7114487125068701696 to 7114487124506460160. Current Snapshot: MvccSnapshot[applied={T|T < 7114487125068701696 or (T in {7114487125068701696})}]
I20250115 10:02:16.095916 7683 master_service.cc:946] ReplaceTablet: received request to replace tablet 0ff0cd8db6b146e3a590c02ce7dac65e from {username='slave'} at 127.0.0.1:34534
I20250115 10:02:16.262166 7683 catalog_manager.cc:5813] T 00000000000000000000000000000000 P 91795ce3770449ca959a107f35c29a16: Sending DeleteTablet for 3 replicas of tablet 0ff0cd8db6b146e3a590c02ce7dac65e
I20250115 10:02:16.263115 7683 catalog_manager.cc:6596] ReplaceTablet: tablet 0ff0cd8db6b146e3a590c02ce7dac65e deleted and replaced by tablet 7f10151581c4422da70cdb198f28766f
I20250115 10:02:16.263904 7804 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ff0cd8db6b146e3a590c02ce7dac65e with delete_type TABLET_DATA_DELETED (replaced by tablet 7f10151581c4422da70cdb198f28766f) from {username='slave'} at 127.0.0.1:57028
I20250115 10:02:16.268153 7933 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ff0cd8db6b146e3a590c02ce7dac65e with delete_type TABLET_DATA_DELETED (replaced by tablet 7f10151581c4422da70cdb198f28766f) from {username='slave'} at 127.0.0.1:32984
I20250115 10:02:16.269344 8062 tablet_service.cc:1514] Processing DeleteTablet for tablet 0ff0cd8db6b146e3a590c02ce7dac65e with delete_type TABLET_DATA_DELETED (replaced by tablet 7f10151581c4422da70cdb198f28766f) from {username='slave'} at 127.0.0.1:59858
I20250115 10:02:16.287904 7932 tablet_service.cc:1467] Processing CreateTablet for tablet 7f10151581c4422da70cdb198f28766f (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250115 10:02:16.288394 7932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f10151581c4422da70cdb198f28766f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:16.292513 8064 tablet_service.cc:1467] Processing CreateTablet for tablet 7f10151581c4422da70cdb198f28766f (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250115 10:02:16.292984 8064 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f10151581c4422da70cdb198f28766f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:16.297076 7801 tablet_service.cc:1467] Processing CreateTablet for tablet 7f10151581c4422da70cdb198f28766f (DEFAULT_TABLE table=test-workload [id=b10f876ec31d44289cc16f84063577e2]), partition=RANGE (key) PARTITION 536870911 <= VALUES < 1073741822
I20250115 10:02:16.297510 7801 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f10151581c4422da70cdb198f28766f. 1 dirs total, 0 dirs full, 0 dirs failed
I20250115 10:02:16.356144 8253 tablet_bootstrap.cc:492] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d: Bootstrap starting.
I20250115 10:02:16.366683 8253 tablet_bootstrap.cc:654] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:16.370800 7683 master_service.cc:946] ReplaceTablet: received request to replace tablet 0ff0cd8db6b146e3a590c02ce7dac65e from {username='slave'} at 127.0.0.1:34534
/home/jenkins-slave/workspace/build_and_test_flaky@2/src/kudu/integration-tests/replace_tablet-itest.cc:133: Failure
Failed
Bad status: Not found: Tablet 0ff0cd8db6b146e3a590c02ce7dac65e already deleted
I20250115 10:02:16.376808 8253 tablet_bootstrap.cc:492] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d: No bootstrap required, opened a new log
I20250115 10:02:16.377085 8253 ts_tablet_manager.cc:1397] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.000s
I20250115 10:02:16.378036 8253 raft_consensus.cc:357] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } }
I20250115 10:02:16.378422 8253 raft_consensus.cc:383] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:16.378580 8253 raft_consensus.cc:738] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 236c0d893ac742beb1c31a07f33ce84d, State: Initialized, Role: FOLLOWER
I20250115 10:02:16.378898 8253 consensus_queue.cc:260] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } }
I20250115 10:02:16.381177 8253 ts_tablet_manager.cc:1428] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20250115 10:02:16.394407 8251 tablet_bootstrap.cc:492] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5: Bootstrap starting.
I20250115 10:02:16.395984 8251 tablet_bootstrap.cc:654] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:16.397143 8250 tablet_replica.cc:331] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: stopping tablet replica
I20250115 10:02:16.397610 8249 tablet_replica.cc:331] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: stopping tablet replica
I20250115 10:02:16.398276 8250 raft_consensus.cc:2238] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 10:02:16.398881 8249 raft_consensus.cc:2238] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Raft consensus shutting down.
I20250115 10:02:16.399356 8249 raft_consensus.cc:2267] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 10:02:16.400171 8248 tablet_replica.cc:331] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: stopping tablet replica
I20250115 10:02:16.400701 8250 raft_consensus.cc:2267] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 10:02:16.401594 8248 raft_consensus.cc:2238] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 1 LEADER]: Raft consensus shutting down.
I20250115 10:02:16.402375 8248 raft_consensus.cc:2267] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20250115 10:02:16.403453 8251 tablet_bootstrap.cc:492] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5: No bootstrap required, opened a new log
I20250115 10:02:16.405058 8250 ts_tablet_manager.cc:1905] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 10:02:16.405439 8252 tablet_bootstrap.cc:492] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078: Bootstrap starting.
I20250115 10:02:16.409482 8252 tablet_bootstrap.cc:654] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078: Neither blocks nor log segments found. Creating new log.
I20250115 10:02:16.408660 8248 ts_tablet_manager.cc:1905] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 10:02:16.411345 7644 external_mini_cluster-itest-base.cc:80] Found fatal failure
I20250115 10:02:16.412946 7644 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 0 with UUID bf32bd013c6048c584442f708b0427a5 and pid 7742
I20250115 10:02:16.414613 8250 ts_tablet_manager.cc:1918] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.224
I20250115 10:02:16.414803 8250 log.cc:1198] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: Deleting WAL directory at /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-2/wal/wals/0ff0cd8db6b146e3a590c02ce7dac65e
I20250115 10:02:16.415359 8250 ts_tablet_manager.cc:1939] T 0ff0cd8db6b146e3a590c02ce7dac65e P ad9d5d231fad4f0290f5d987cea32078: Deleting consensus metadata
I20250115 10:02:16.416934 7668 catalog_manager.cc:4872] TS ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3:34549): tablet 0ff0cd8db6b146e3a590c02ce7dac65e (table test-workload [id=b10f876ec31d44289cc16f84063577e2]) successfully deleted
I20250115 10:02:16.417409 8251 ts_tablet_manager.cc:1397] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.000s
I20250115 10:02:16.418778 8249 ts_tablet_manager.cc:1905] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20250115 10:02:16.418391 8251 raft_consensus.cc:357] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } }
I20250115 10:02:16.420745 8251 raft_consensus.cc:383] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:16.421181 8251 raft_consensus.cc:738] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf32bd013c6048c584442f708b0427a5, State: Initialized, Role: FOLLOWER
I20250115 10:02:16.421501 8251 consensus_queue.cc:260] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } }
I20250115 10:02:16.422235 8251 ts_tablet_manager.cc:1428] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20250115 10:02:16.424911 8248 ts_tablet_manager.cc:1918] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.224
I20250115 10:02:16.425123 8248 log.cc:1198] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: Deleting WAL directory at /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-0/wal/wals/0ff0cd8db6b146e3a590c02ce7dac65e
I20250115 10:02:16.425750 8248 ts_tablet_manager.cc:1939] T 0ff0cd8db6b146e3a590c02ce7dac65e P bf32bd013c6048c584442f708b0427a5: Deleting consensus metadata
I20250115 10:02:16.427680 8249 ts_tablet_manager.cc:1918] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.224
I20250115 10:02:16.427838 8249 log.cc:1198] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: Deleting WAL directory at /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0/minicluster-data/ts-1/wal/wals/0ff0cd8db6b146e3a590c02ce7dac65e
I20250115 10:02:16.428433 8249 ts_tablet_manager.cc:1939] T 0ff0cd8db6b146e3a590c02ce7dac65e P 236c0d893ac742beb1c31a07f33ce84d: Deleting consensus metadata
I20250115 10:02:16.428752 7668 catalog_manager.cc:4872] TS bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903): tablet 0ff0cd8db6b146e3a590c02ce7dac65e (table test-workload [id=b10f876ec31d44289cc16f84063577e2]) successfully deleted
I20250115 10:02:16.430003 7670 catalog_manager.cc:4872] TS 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047): tablet 0ff0cd8db6b146e3a590c02ce7dac65e (table test-workload [id=b10f876ec31d44289cc16f84063577e2]) successfully deleted
I20250115 10:02:16.430647 8252 tablet_bootstrap.cc:492] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078: No bootstrap required, opened a new log
I20250115 10:02:16.430783 8252 ts_tablet_manager.cc:1397] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078: Time spent bootstrapping tablet: real 0.025s user 0.007s sys 0.014s
I20250115 10:02:16.431414 8252 raft_consensus.cc:357] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } }
I20250115 10:02:16.431582 8252 raft_consensus.cc:383] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20250115 10:02:16.431663 8252 raft_consensus.cc:738] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad9d5d231fad4f0290f5d987cea32078, State: Initialized, Role: FOLLOWER
I20250115 10:02:16.431881 8252 consensus_queue.cc:260] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } }
I20250115 10:02:16.432430 8252 ts_tablet_manager.cc:1428] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20250115 10:02:16.488227 8254 raft_consensus.cc:491] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20250115 10:02:16.488401 8254 raft_consensus.cc:513] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } }
I20250115 10:02:16.489184 8254 leader_election.cc:290] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903), 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047)
I20250115 10:02:16.489754 7822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f10151581c4422da70cdb198f28766f" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf32bd013c6048c584442f708b0427a5" is_pre_election: true
I20250115 10:02:16.489821 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f10151581c4422da70cdb198f28766f" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d" is_pre_election: true
I20250115 10:02:16.489989 7822 raft_consensus.cc:2463] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9d5d231fad4f0290f5d987cea32078 in term 0.
I20250115 10:02:16.490105 7955 raft_consensus.cc:2463] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad9d5d231fad4f0290f5d987cea32078 in term 0.
I20250115 10:02:16.490479 8019 leader_election.cc:304] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad9d5d231fad4f0290f5d987cea32078, bf32bd013c6048c584442f708b0427a5; no voters:
I20250115 10:02:16.490752 8254 raft_consensus.cc:2798] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20250115 10:02:16.490872 8254 raft_consensus.cc:491] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20250115 10:02:16.490964 8254 raft_consensus.cc:3054] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:16.492980 8254 raft_consensus.cc:513] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } }
I20250115 10:02:16.493564 8254 leader_election.cc:290] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 election: Requested vote from peers bf32bd013c6048c584442f708b0427a5 (127.7.119.1:38903), 236c0d893ac742beb1c31a07f33ce84d (127.7.119.2:39047)
I20250115 10:02:16.494026 7822 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f10151581c4422da70cdb198f28766f" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf32bd013c6048c584442f708b0427a5"
I20250115 10:02:16.494108 7955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f10151581c4422da70cdb198f28766f" candidate_uuid: "ad9d5d231fad4f0290f5d987cea32078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "236c0d893ac742beb1c31a07f33ce84d"
I20250115 10:02:16.494271 7822 raft_consensus.cc:3054] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5 [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:16.494292 7955 raft_consensus.cc:3054] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d [term 0 FOLLOWER]: Advancing to term 1
I20250115 10:02:16.497148 7955 raft_consensus.cc:2463] T 7f10151581c4422da70cdb198f28766f P 236c0d893ac742beb1c31a07f33ce84d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9d5d231fad4f0290f5d987cea32078 in term 1.
I20250115 10:02:16.497150 7822 raft_consensus.cc:2463] T 7f10151581c4422da70cdb198f28766f P bf32bd013c6048c584442f708b0427a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad9d5d231fad4f0290f5d987cea32078 in term 1.
I20250115 10:02:16.497570 8019 leader_election.cc:304] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ad9d5d231fad4f0290f5d987cea32078, bf32bd013c6048c584442f708b0427a5; no voters:
I20250115 10:02:16.497840 8254 raft_consensus.cc:2798] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 1 FOLLOWER]: Leader election won for term 1
I20250115 10:02:16.498009 8254 raft_consensus.cc:695] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [term 1 LEADER]: Becoming Leader. State: Replica: ad9d5d231fad4f0290f5d987cea32078, State: Running, Role: LEADER
I20250115 10:02:16.498297 8254 consensus_queue.cc:237] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } }
I20250115 10:02:16.501366 7683 catalog_manager.cc:5526] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad9d5d231fad4f0290f5d987cea32078 (127.7.119.3). New cstate: current_term: 1 leader_uuid: "ad9d5d231fad4f0290f5d987cea32078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad9d5d231fad4f0290f5d987cea32078" member_type: VOTER last_known_addr { host: "127.7.119.3" port: 34549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 } health_report { overall_health: UNKNOWN } } }
************************ BEGIN STACKS **************************
[New LWP 7743]
[New LWP 7744]
[New LWP 7745]
[New LWP 7746]
[New LWP 7752]
[New LWP 7753]
[New LWP 7754]
[New LWP 7757]
[New LWP 7758]
[New LWP 7759]
[New LWP 7760]
[New LWP 7761]
[New LWP 7762]
[New LWP 7763]
[New LWP 7764]
[New LWP 7765]
[New LWP 7766]
[New LWP 7767]
[New LWP 7768]
[New LWP 7769]
[New LWP 7770]
[New LWP 7771]
[New LWP 7772]
[New LWP 7773]
[New LWP 7774]
[New LWP 7775]
[New LWP 7776]
[New LWP 7777]
[New LWP 7778]
[New LWP 7779]
[New LWP 7780]
[New LWP 7781]
[New LWP 7782]
[New LWP 7783]
[New LWP 7784]
[New LWP 7785]
[New LWP 7786]
[New LWP 7787]
[New LWP 7788]
[New LWP 7789]
[New LWP 7790]
[New LWP 7791]
[New LWP 7792]
[New LWP 7793]
[New LWP 7794]
[New LWP 7795]
[New LWP 7796]
[New LWP 7797]
[New LWP 7798]
[New LWP 7799]
[New LWP 7800]
[New LWP 7801]
[New LWP 7802]
[New LWP 7803]
[New LWP 7804]
[New LWP 7805]
[New LWP 7806]
[New LWP 7807]
[New LWP 7808]
[New LWP 7809]
[New LWP 7810]
[New LWP 7811]
[New LWP 7812]
[New LWP 7813]
[New LWP 7814]
[New LWP 7815]
[New LWP 7816]
[New LWP 7817]
[New LWP 7818]
[New LWP 7819]
[New LWP 7820]
[New LWP 7821]
[New LWP 7822]
[New LWP 7823]
[New LWP 7824]
[New LWP 7825]
[New LWP 7826]
[New LWP 7827]
[New LWP 7828]
[New LWP 7829]
[New LWP 7830]
[New LWP 7831]
[New LWP 7832]
[New LWP 7833]
[New LWP 7834]
[New LWP 7835]
[New LWP 7836]
[New LWP 7837]
[New LWP 7838]
[New LWP 7839]
[New LWP 7840]
[New LWP 7841]
[New LWP 7842]
[New LWP 7843]
[New LWP 7844]
[New LWP 7845]
[New LWP 7846]
[New LWP 7847]
[New LWP 7848]
[New LWP 7849]
[New LWP 7850]
[New LWP 7851]
[New LWP 7852]
[New LWP 7853]
[New LWP 7854]
[New LWP 7855]
[New LWP 7856]
[New LWP 7857]
[New LWP 7858]
[New LWP 7859]
[New LWP 7860]
[New LWP 7861]
[New LWP 7862]
[New LWP 7863]
[New LWP 7864]
[New LWP 7865]
[New LWP 7866]
[New LWP 7867]
[New LWP 7868]
[New LWP 7869]
[New LWP 7870]
[New LWP 8183]
[New LWP 8219]
[New LWP 8220]
[New LWP 8234]
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
0x00007fae74dded50 in ?? ()
Id Target Id Frame
* 1 LWP 7742 "kudu" 0x00007fae74dded50 in ?? ()
2 LWP 7743 "kudu" 0x00007fae74ddafb9 in ?? ()
3 LWP 7744 "kudu" 0x00007fae74ddafb9 in ?? ()
4 LWP 7745 "kudu" 0x00007fae74ddafb9 in ?? ()
5 LWP 7746 "kernel-watcher-" 0x00007fae74ddafb9 in ?? ()
6 LWP 7752 "ntp client-7752" 0x00007fae74dde9e2 in ?? ()
7 LWP 7753 "file cache-evic" 0x00007fae74ddafb9 in ?? ()
8 LWP 7754 "sq_acceptor" 0x00007fae6db3ecb9 in ?? ()
9 LWP 7757 "rpc reactor-775" 0x00007fae6db4ba47 in ?? ()
10 LWP 7758 "rpc reactor-775" 0x00007fae6db4ba47 in ?? ()
11 LWP 7759 "rpc reactor-775" 0x00007fae6db4ba47 in ?? ()
12 LWP 7760 "rpc reactor-776" 0x00007fae6db4ba47 in ?? ()
13 LWP 7761 "MaintenanceMgr " 0x00007fae74ddaad3 in ?? ()
14 LWP 7762 "txn-status-mana" 0x00007fae74ddafb9 in ?? ()
15 LWP 7763 "collect_and_rem" 0x00007fae74ddafb9 in ?? ()
16 LWP 7764 "tc-session-exp-" 0x00007fae74ddafb9 in ?? ()
17 LWP 7765 "rpc worker-7765" 0x00007fae74ddaad3 in ?? ()
18 LWP 7766 "rpc worker-7766" 0x00007fae74ddaad3 in ?? ()
19 LWP 7767 "rpc worker-7767" 0x00007fae74ddaad3 in ?? ()
20 LWP 7768 "rpc worker-7768" 0x00007fae74ddaad3 in ?? ()
21 LWP 7769 "rpc worker-7769" 0x00007fae74ddaad3 in ?? ()
22 LWP 7770 "rpc worker-7770" 0x00007fae74ddaad3 in ?? ()
23 LWP 7771 "rpc worker-7771" 0x00007fae74ddaad3 in ?? ()
24 LWP 7772 "rpc worker-7772" 0x00007fae74ddaad3 in ?? ()
25 LWP 7773 "rpc worker-7773" 0x00007fae74ddaad3 in ?? ()
26 LWP 7774 "rpc worker-7774" 0x00007fae74ddaad3 in ?? ()
27 LWP 7775 "rpc worker-7775" 0x00007fae74ddaad3 in ?? ()
28 LWP 7776 "rpc worker-7776" 0x00007fae74ddaad3 in ?? ()
29 LWP 7777 "rpc worker-7777" 0x00007fae74ddaad3 in ?? ()
30 LWP 7778 "rpc worker-7778" 0x00007fae74ddaad3 in ?? ()
31 LWP 7779 "rpc worker-7779" 0x00007fae74ddaad3 in ?? ()
32 LWP 7780 "rpc worker-7780" 0x00007fae74ddaad3 in ?? ()
33 LWP 7781 "rpc worker-7781" 0x00007fae74ddaad3 in ?? ()
34 LWP 7782 "rpc worker-7782" 0x00007fae74ddaad3 in ?? ()
35 LWP 7783 "rpc worker-7783" 0x00007fae74ddaad3 in ?? ()
36 LWP 7784 "rpc worker-7784" 0x00007fae74ddaad3 in ?? ()
37 LWP 7785 "rpc worker-7785" 0x00007fae74ddaad3 in ?? ()
38 LWP 7786 "rpc worker-7786" 0x00007fae74ddaad3 in ?? ()
39 LWP 7787 "rpc worker-7787" 0x00007fae74ddaad3 in ?? ()
40 LWP 7788 "rpc worker-7788" 0x00007fae74ddaad3 in ?? ()
41 LWP 7789 "rpc worker-7789" 0x00007fae74ddaad3 in ?? ()
42 LWP 7790 "rpc worker-7790" 0x00007fae74ddaad3 in ?? ()
43 LWP 7791 "rpc worker-7791" 0x00007fae74ddaad3 in ?? ()
44 LWP 7792 "rpc worker-7792" 0x00007fae74ddaad3 in ?? ()
45 LWP 7793 "rpc worker-7793" 0x00007fae74ddaad3 in ?? ()
46 LWP 7794 "rpc worker-7794" 0x00007fae74ddaad3 in ?? ()
47 LWP 7795 "rpc worker-7795" 0x00007fae74ddaad3 in ?? ()
48 LWP 7796 "rpc worker-7796" 0x00007fae74ddaad3 in ?? ()
49 LWP 7797 "rpc worker-7797" 0x00007fae74ddaad3 in ?? ()
50 LWP 7798 "rpc worker-7798" 0x00007fae74ddaad3 in ?? ()
51 LWP 7799 "rpc worker-7799" 0x00007fae74ddaad3 in ?? ()
52 LWP 7800 "rpc worker-7800" 0x00007fae74ddaad3 in ?? ()
53 LWP 7801 "rpc worker-7801" 0x00007fae74ddaad3 in ?? ()
54 LWP 7802 "rpc worker-7802" 0x00007fae74ddaad3 in ?? ()
55 LWP 7803 "rpc worker-7803" 0x00007fae74ddaad3 in ?? ()
56 LWP 7804 "rpc worker-7804" 0x00007fae74ddaad3 in ?? ()
57 LWP 7805 "rpc worker-7805" 0x00007fae74ddaad3 in ?? ()
58 LWP 7806 "rpc worker-7806" 0x00007fae74ddaad3 in ?? ()
59 LWP 7807 "rpc worker-7807" 0x00007fae74ddaad3 in ?? ()
60 LWP 7808 "rpc worker-7808" 0x00007fae74ddaad3 in ?? ()
61 LWP 7809 "rpc worker-7809" 0x00007fae74ddaad3 in ?? ()
62 LWP 7810 "rpc worker-7810" 0x00007fae74ddaad3 in ?? ()
63 LWP 7811 "rpc worker-7811" 0x00007fae74ddaad3 in ?? ()
64 LWP 7812 "rpc worker-7812" 0x00007fae74ddaad3 in ?? ()
65 LWP 7813 "rpc worker-7813" 0x00007fae74ddaad3 in ?? ()
66 LWP 7814 "rpc worker-7814" 0x00007fae74ddaad3 in ?? ()
67 LWP 7815 "rpc worker-7815" 0x00007fae74ddaad3 in ?? ()
68 LWP 7816 "rpc worker-7816" 0x00007fae74ddaad3 in ?? ()
69 LWP 7817 "rpc worker-7817" 0x00007fae74ddaad3 in ?? ()
70 LWP 7818 "rpc worker-7818" 0x00007fae74ddaad3 in ?? ()
71 LWP 7819 "rpc worker-7819" 0x00007fae74ddaad3 in ?? ()
72 LWP 7820 "rpc worker-7820" 0x00007fae74ddaad3 in ?? ()
73 LWP 7821 "rpc worker-7821" 0x00007fae74ddaad3 in ?? ()
74 LWP 7822 "rpc worker-7822" 0x00007fae74ddaad3 in ?? ()
75 LWP 7823 "rpc worker-7823" 0x00007fae74ddaad3 in ?? ()
76 LWP 7824 "rpc worker-7824" 0x00007fae74ddaad3 in ?? ()
77 LWP 7825 "rpc worker-7825" 0x00007fae74ddaad3 in ?? ()
78 LWP 7826 "rpc worker-7826" 0x00007fae74ddaad3 in ?? ()
79 LWP 7827 "rpc worker-7827" 0x00007fae74ddaad3 in ?? ()
80 LWP 7828 "rpc worker-7828" 0x00007fae74ddaad3 in ?? ()
81 LWP 7829 "rpc worker-7829" 0x00007fae74ddaad3 in ?? ()
82 LWP 7830 "rpc worker-7830" 0x00007fae74ddaad3 in ?? ()
83 LWP 7831 "rpc worker-7831" 0x00007fae74ddaad3 in ?? ()
84 LWP 7832 "rpc worker-7832" 0x00007fae74ddaad3 in ?? ()
85 LWP 7833 "rpc worker-7833" 0x00007fae74ddaad3 in ?? ()
86 LWP 7834 "rpc worker-7834" 0x00007fae74ddaad3 in ?? ()
87 LWP 7835 "rpc worker-7835" 0x00007fae74ddaad3 in ?? ()
88 LWP 7836 "rpc worker-7836" 0x00007fae74ddaad3 in ?? ()
89 LWP 7837 "rpc worker-7837" 0x00007fae74ddaad3 in ?? ()
90 LWP 7838 "rpc worker-7838" 0x00007fae74ddaad3 in ?? ()
91 LWP 7839 "rpc worker-7839" 0x00007fae74ddaad3 in ?? ()
92 LWP 7840 "rpc worker-7840" 0x00007fae74ddaad3 in ?? ()
93 LWP 7841 "rpc worker-7841" 0x00007fae74ddaad3 in ?? ()
94 LWP 7842 "rpc worker-7842" 0x00007fae74ddaad3 in ?? ()
95 LWP 7843 "rpc worker-7843" 0x00007fae74ddaad3 in ?? ()
96 LWP 7844 "rpc worker-7844" 0x00007fae74ddaad3 in ?? ()
97 LWP 7845 "rpc worker-7845" 0x00007fae74ddaad3 in ?? ()
98 LWP 7846 "rpc worker-7846" 0x00007fae74ddaad3 in ?? ()
99 LWP 7847 "rpc worker-7847" 0x00007fae74ddaad3 in ?? ()
100 LWP 7848 "rpc worker-7848" 0x00007fae74ddaad3 in ?? ()
101 LWP 7849 "rpc worker-7849" 0x00007fae74ddaad3 in ?? ()
102 LWP 7850 "rpc worker-7850" 0x00007fae74ddaad3 in ?? ()
103 LWP 7851 "rpc worker-7851" 0x00007fae74ddaad3 in ?? ()
104 LWP 7852 "rpc worker-7852" 0x00007fae74ddaad3 in ?? ()
105 LWP 7853 "rpc worker-7853" 0x00007fae74ddaad3 in ?? ()
106 LWP 7854 "rpc worker-7854" 0x00007fae74ddaad3 in ?? ()
107 LWP 7855 "rpc worker-7855" 0x00007fae74ddaad3 in ?? ()
108 LWP 7856 "rpc worker-7856" 0x00007fae74ddaad3 in ?? ()
109 LWP 7857 "rpc worker-7857" 0x00007fae74ddaad3 in ?? ()
110 LWP 7858 "rpc worker-7858" 0x00007fae74ddaad3 in ?? ()
111 LWP 7859 "rpc worker-7859" 0x00007fae74ddaad3 in ?? ()
112 LWP 7860 "rpc worker-7860" 0x00007fae74ddaad3 in ?? ()
113 LWP 7861 "rpc worker-7861" 0x00007fae74ddaad3 in ?? ()
114 LWP 7862 "rpc worker-7862" 0x00007fae74ddaad3 in ?? ()
115 LWP 7863 "rpc worker-7863" 0x00007fae74ddaad3 in ?? ()
116 LWP 7864 "rpc worker-7864" 0x00007fae74ddaad3 in ?? ()
117 LWP 7865 "diag-logger-786" 0x00007fae74ddafb9 in ?? ()
118 LWP 7866 "result-tracker-" 0x00007fae74ddafb9 in ?? ()
119 LWP 7867 "excess-log-dele" 0x00007fae74ddafb9 in ?? ()
120 LWP 7868 "acceptor-7868" 0x00007fae6db4d0c7 in ?? ()
121 LWP 7869 "heartbeat-7869" 0x00007fae74ddafb9 in ?? ()
122 LWP 7870 "maintenance_sch" 0x00007fae74ddafb9 in ?? ()
123 LWP 8183 "wal-append [wor" 0x00007fae74ddafb9 in ?? ()
124 LWP 8219 "wal-append [wor" 0x00007fae74ddafb9 in ?? ()
125 LWP 8220 "wal-append [wor" 0x00007fae74ddafb9 in ?? ()
126 LWP 8234 "raft [worker]-8" 0x00007fae74ddafb9 in ?? ()
Thread 126 (LWP 8234):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000183 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007fae26554b24 in ?? ()
#5 0x00007fae265547b0 in ?? ()
#6 0x0000000000000307 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007fae265547d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 125 (LWP 8220):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x00006020001bd438 in ?? ()
#2 0x0000000000000140 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061a000046fb0 in ?? ()
#5 0x00007fae203e5090 in ?? ()
#6 0x0000000000000280 in ?? ()
#7 0x0000000000000000 in ?? ()
I20250115 10:02:16.957664 8254 consensus_queue.cc:1035] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "236c0d893ac742beb1c31a07f33ce84d" member_type: VOTER last_known_addr { host: "127.7.119.2" port: 39047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Thread 124 (LWP 8219):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x000060200019dc98 in ?? ()
#2 0x0000000000000178 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061a0000457b0 in ?? ()
#5 0x00007fae20bf4090 in ?? ()
#6 0x00000000000002f0 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 123 (LWP 8183):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x00006020001cc058 in ?? ()
#2 0x00000000000001a4 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061a0000421b0 in ?? ()
#5 0x00007fae24cdd090 in ?? ()
#6 0x0000000000000348 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 122 (LWP 7870):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000014 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007fae29e9d550 in ?? ()
#6 0x0000000000000028 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 7869):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x48e3b5d9e7beadbb in ?? ()
#2 0x000000000000000c in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000613000019400 in ?? ()
#5 0x00007fae2a6b55b0 in ?? ()
#6 0x0000000000000018 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 120 (LWP 7868):
#0 0x00007fae6db4d0c7 in ?? ()
#1 0x00006140000222a8 in ?? ()
#2 0x00007fae2aedcd50 in ?? ()
#3 0x00007fae2aedcd80 in ?? ()
#4 0x00007fae2aedce80 in ?? ()
#5 0x00007fae2aedcd70 in ?? ()
#6 0x00007fae2aedcde0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007fae2aedcf00 in ?? ()
#11 0x00007fae2aedc63c in ?? ()
#12 0x000000322aedc5b0 in ?? ()
#13 0x00007fae2a6e0000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 7867):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x00007fae2b6f6f00 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 7866):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x00007fae2bf0f0c0 in ?? ()
#2 0x0000000000000005 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000879b0 in ?? ()
#5 0x00007fae2bf0f0b0 in ?? ()
#6 0x000000000000000a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 7865):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 7864):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 115 (LWP 7863):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 114 (LWP 7862):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 7861):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 7860):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 7859):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 7858):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 7857):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 7856):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 7855):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 7854):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 7853):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 7852):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 7851):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 7850):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 7849):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 7848):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 7847):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 7846):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 7845):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 7844):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 7843):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 7842):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 7841):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 7840):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 7839):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 7838):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 7837):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 7836):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 7835):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 7834):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 7833):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 7832):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 7831):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 7830):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 7829):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 7828):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 7827):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 7826):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 7825):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 7824):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x00000000000000b1 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001ad08c in ?? ()
#4 0x00007fae412ffe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae412ffe70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001ad040 in ?? ()
#9 0x00007fae74dda770 in ?? ()
#10 0x00007fae412ffe70 in ?? ()
#11 0x00000ff5c825ffd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 75 (LWP 7823):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000094 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007fae41b17e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae41b17e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 7822):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x00000000000000b2 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a0088 in ?? ()
#4 0x00007fae4232fe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae4232fe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 73 (LWP 7821):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 72 (LWP 7820):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 7819):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 7818):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 7817):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 7816):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 7815):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 7814):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 7813):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 7812):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 7811):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 7810):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 7809):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 7808):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 7807):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 7806):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 7805):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 7804):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000126838 in ?? ()
#4 0x00007fae4b4e0e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae4b4e0e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 55 (LWP 7803):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007fae4bcf8e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae4bcf8e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007fae74dda770 in ?? ()
#10 0x00007fae4bcf8e70 in ?? ()
#11 0x00000ff5c979f1d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 54 (LWP 7802):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00011d04c in ?? ()
#4 0x00007fae4c510e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae4c510e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011d000 in ?? ()
#9 0x00007fae74dda770 in ?? ()
#10 0x00007fae4c510e70 in ?? ()
#11 0x00000ff5c98a21d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 53 (LWP 7801):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000116848 in ?? ()
#4 0x00007fae4cd28e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae4cd28e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 52 (LWP 7800):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 7799):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 7798):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 7797):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 7796):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 7795):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 7794):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 7793):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 7792):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 7791):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 7790):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 7789):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 7788):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 7787):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 7786):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 7785):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 7784):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x00000000000000b7 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007fae556c1e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae556c1e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007fae74dda770 in ?? ()
#10 0x00007fae556c1e70 in ?? ()
#11 0x00000ff5caad83d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 7783):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007fae55ed9e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae55ed9e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007fae74dda770 in ?? ()
#10 0x00007fae55ed9e70 in ?? ()
#11 0x00000ff5cabdb3d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 34 (LWP 7782):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000004 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0000967f8 in ?? ()
#4 0x00007fae566f1e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae566f1e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 33 (LWP 7781):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000053 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008fffc in ?? ()
#4 0x00007fae56f09e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae56f09e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00008ffb0 in ?? ()
#9 0x00007fae74dda770 in ?? ()
#10 0x00007fae56f09e70 in ?? ()
#11 0x00000ff5cade13d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 32 (LWP 7780):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000069 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008d00c in ?? ()
#4 0x00007fae57721e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae57721e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00008cfc0 in ?? ()
#9 0x00007fae74dda770 in ?? ()
#10 0x00007fae57721e70 in ?? ()
#11 0x00000ff5caee43d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 31 (LWP 7779):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000044 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000086808 in ?? ()
#4 0x00007fae57f39e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae57f39e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 30 (LWP 7778):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00008000c in ?? ()
#4 0x00007fae58751e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae58751e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00007ffc0 in ?? ()
#9 0x00007fae74dda770 in ?? ()
#10 0x00007fae58751e70 in ?? ()
#11 0x00000ff5cb0ea3d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 29 (LWP 7777):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00007681c in ?? ()
#4 0x00007fae58f69e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007fae58f69e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000767d0 in ?? ()
#9 0x00007fae74dda770 in ?? ()
#10 0x00007fae58f69e70 in ?? ()
#11 0x00000ff5cb1ed3d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 28 (LWP 7776):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 7775):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 7774):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 7773):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 7772):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 7771):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 7770):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 7769):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 7768):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 7767):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 7766):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 7765):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 7764):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x00007fae5f8d6c80 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 15 (LWP 7763):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 7762):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x00007fae609161c0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 13 (LWP 7761):
#0 0x00007fae74ddaad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 7760):
#0 0x00007fae6db4ba47 in ?? ()
#1 0x00007fae619572e0 in ?? ()
#2 0x000061a00000f680 in ?? ()
#3 0x00007fae619572d0 in ?? ()
#4 0x00007fae619574e0 in ?? ()
#5 0x00007fae61957320 in ?? ()
#6 0x0000614000022498 in ?? ()
#7 0x00007fae619573a0 in ?? ()
#8 0x00007fae701ab25d in ?? ()
#9 0x3fb970db9b7dc000 in ?? ()
#10 0x000061a00000f680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f680 in ?? ()
#13 0x00000000fd000000 in ?? ()
#14 0x00007fae00000000 in ?? ()
#15 0x41d9e1e0d82ea201 in ?? ()
#16 0x00000ff64c322e74 in ?? ()
#17 0x00007fae61957380 in ?? ()
#18 0x00007fae701afba3 in ?? ()
#19 0x00007fae61957350 in ?? ()
#20 0x3fb970db9b7dc000 in ?? ()
#21 0x00000000619573a0 in ?? ()
#22 0x000061a00000f680 in ?? ()
#23 0x0000614000022498 in ?? ()
#24 0x3fb970db9b7dc000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 7759):
#0 0x00007fae6db4ba47 in ?? ()
#1 0x00007fae6216f2e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007fae6216f2d0 in ?? ()
#4 0x00007fae6216f4e0 in ?? ()
#5 0x00007fae6216f320 in ?? ()
#6 0x0000614000022298 in ?? ()
#7 0x00007fae6216f3a0 in ?? ()
#8 0x00007fae701ab25d in ?? ()
#9 0x3fb95eaa33af4000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x00000000f7000000 in ?? ()
#14 0x00007fae00000000 in ?? ()
#15 0x41d9e1e0d82ea204 in ?? ()
#16 0x00000ff64c425e74 in ?? ()
#17 0x00007fae6216f380 in ?? ()
#18 0x00007fae701afba3 in ?? ()
#19 0x00007fae6216f350 in ?? ()
#20 0x3fb95eaa33af4000 in ?? ()
#21 0x000000006216f3a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000022298 in ?? ()
#24 0x3fb95eaa33af4000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 7758):
#0 0x00007fae6db4ba47 in ?? ()
#1 0x00007fae629862e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007fae629862d0 in ?? ()
#4 0x00007fae629864e0 in ?? ()
#5 0x00007fae62986320 in ?? ()
#6 0x0000614000022098 in ?? ()
#7 0x00007fae629863a0 in ?? ()
#8 0x00007fae701ab25d in ?? ()
#9 0x3fb973a4632d4000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x00000000e3000000 in ?? ()
#14 0x00007fae00000000 in ?? ()
#15 0x41d9e1e0d82ea204 in ?? ()
#16 0x00000ff64c528c74 in ?? ()
#17 0x00007fae62986380 in ?? ()
#18 0x00007fae701afba3 in ?? ()
#19 0x00007fae62986350 in ?? ()
#20 0x3fb973a4632d4000 in ?? ()
#21 0x00000000629863a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000022098 in ?? ()
#24 0x3fb973a4632d4000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 7757):
#0 0x00007fae6db4ba47 in ?? ()
#1 0x00007fae6499d2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007fae6499d2d0 in ?? ()
#4 0x00007fae6499d4e0 in ?? ()
#5 0x00007fae6499d320 in ?? ()
#6 0x0000614000021e98 in ?? ()
#7 0x00007fae6499d3a0 in ?? ()
#8 0x00007fae701ab25d in ?? ()
#9 0x3fb951a40c89c000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x0000000053000000 in ?? ()
#14 0x00007fae00000000 in ?? ()
#15 0x41d9e1e0d82ea202 in ?? ()
#16 0x00000ff64c92ba74 in ?? ()
#17 0x00007fae6499d380 in ?? ()
#18 0x00007fae701afba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 7754):
#0 0x00007fae6db3ecb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007fae663b7788 in ?? ()
#3 0x0000602000017790 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 7753):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x00007fae65bb6080 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 7752):
#0 0x00007fae74dde9e2 in ?? ()
#1 0x00007fae653b3b60 in ?? ()
#2 0x00007fae653b3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007fae653b3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007fae653b3c10 in ?? ()
#8 0x0000611000088950 in ?? ()
#9 0x00007fae653b3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007fae7a702f0c in ?? ()
#12 0x00007fae7a6f27b9 in ?? ()
#13 0x00007fae7a6f6da6 in ?? ()
#14 0x00007fae7a6fb7e0 in ?? ()
#15 0x00007fae7a6fb609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007fae719f298e in ?? ()
#18 0x00007fae74dd46db in ?? ()
#19 0x00000ff64ca6e67c in ?? ()
#20 0x00007fae653b3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 7746):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x00007fae673b8c40 in ?? ()
#2 0x000000000000001a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007fae673b8c30 in ?? ()
#6 0x0000000000000034 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 7745):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 7744):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 7743):
#0 0x00007fae74ddafb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007fae68bbc410 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 7742):
#0 0x00007fae74dded50 in ?? ()
#1 0x0000000000000000 in ?? ()
I20250115 10:02:17.006750 8254 consensus_queue.cc:1035] T 7f10151581c4422da70cdb198f28766f P ad9d5d231fad4f0290f5d987cea32078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf32bd013c6048c584442f708b0427a5" member_type: VOTER last_known_addr { host: "127.7.119.1" port: 38903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
************************* END STACKS ***************************
I20250115 10:02:17.016767 7644 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 1 with UUID 236c0d893ac742beb1c31a07f33ce84d and pid 7873
************************ BEGIN STACKS **************************
[New LWP 7874]
[New LWP 7875]
[New LWP 7876]
[New LWP 7877]
[New LWP 7883]
[New LWP 7884]
[New LWP 7885]
[New LWP 7888]
[New LWP 7889]
[New LWP 7890]
[New LWP 7891]
[New LWP 7892]
[New LWP 7893]
[New LWP 7894]
[New LWP 7895]
[New LWP 7896]
[New LWP 7897]
[New LWP 7898]
[New LWP 7899]
[New LWP 7900]
[New LWP 7901]
[New LWP 7902]
[New LWP 7903]
[New LWP 7904]
[New LWP 7905]
[New LWP 7906]
[New LWP 7907]
[New LWP 7908]
[New LWP 7909]
[New LWP 7910]
[New LWP 7911]
[New LWP 7912]
[New LWP 7913]
[New LWP 7914]
[New LWP 7915]
[New LWP 7916]
[New LWP 7917]
[New LWP 7918]
[New LWP 7919]
[New LWP 7920]
[New LWP 7921]
[New LWP 7922]
[New LWP 7923]
[New LWP 7924]
[New LWP 7925]
[New LWP 7926]
[New LWP 7927]
[New LWP 7928]
[New LWP 7929]
[New LWP 7930]
[New LWP 7931]
[New LWP 7932]
[New LWP 7933]
[New LWP 7934]
[New LWP 7935]
[New LWP 7936]
[New LWP 7937]
[New LWP 7938]
[New LWP 7939]
[New LWP 7940]
[New LWP 7941]
[New LWP 7942]
[New LWP 7943]
[New LWP 7944]
[New LWP 7945]
[New LWP 7946]
[New LWP 7947]
[New LWP 7948]
[New LWP 7949]
[New LWP 7950]
[New LWP 7951]
[New LWP 7952]
[New LWP 7953]
[New LWP 7954]
[New LWP 7955]
[New LWP 7956]
[New LWP 7957]
[New LWP 7958]
[New LWP 7959]
[New LWP 7960]
[New LWP 7961]
[New LWP 7962]
[New LWP 7963]
[New LWP 7964]
[New LWP 7965]
[New LWP 7966]
[New LWP 7967]
[New LWP 7968]
[New LWP 7969]
[New LWP 7970]
[New LWP 7971]
[New LWP 7972]
[New LWP 7973]
[New LWP 7974]
[New LWP 7975]
[New LWP 7976]
[New LWP 7977]
[New LWP 7978]
[New LWP 7979]
[New LWP 7980]
[New LWP 7981]
[New LWP 7982]
[New LWP 7983]
[New LWP 7984]
[New LWP 7985]
[New LWP 7986]
[New LWP 7987]
[New LWP 7988]
[New LWP 7989]
[New LWP 7990]
[New LWP 7991]
[New LWP 7992]
[New LWP 7993]
[New LWP 7994]
[New LWP 7995]
[New LWP 7996]
[New LWP 7997]
[New LWP 7998]
[New LWP 7999]
[New LWP 8000]
[New LWP 8001]
[New LWP 8240]
[New LWP 8267]
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
0x00007f8e27791d50 in ?? ()
Id Target Id Frame
* 1 LWP 7873 "kudu" 0x00007f8e27791d50 in ?? ()
2 LWP 7874 "kudu" 0x00007f8e2778dfb9 in ?? ()
3 LWP 7875 "kudu" 0x00007f8e2778dfb9 in ?? ()
4 LWP 7876 "kudu" 0x00007f8e2778dfb9 in ?? ()
5 LWP 7877 "kernel-watcher-" 0x00007f8e2778dfb9 in ?? ()
6 LWP 7883 "ntp client-7883" 0x00007f8e277919e2 in ?? ()
7 LWP 7884 "file cache-evic" 0x00007f8e2778dfb9 in ?? ()
8 LWP 7885 "sq_acceptor" 0x00007f8e204f1cb9 in ?? ()
9 LWP 7888 "rpc reactor-788" 0x00007f8e204fea47 in ?? ()
10 LWP 7889 "rpc reactor-788" 0x00007f8e204fea47 in ?? ()
11 LWP 7890 "rpc reactor-789" 0x00007f8e204fea47 in ?? ()
12 LWP 7891 "rpc reactor-789" 0x00007f8e204fea47 in ?? ()
13 LWP 7892 "MaintenanceMgr " 0x00007f8e2778dad3 in ?? ()
14 LWP 7893 "txn-status-mana" 0x00007f8e2778dfb9 in ?? ()
15 LWP 7894 "collect_and_rem" 0x00007f8e2778dfb9 in ?? ()
16 LWP 7895 "tc-session-exp-" 0x00007f8e2778dfb9 in ?? ()
17 LWP 7896 "rpc worker-7896" 0x00007f8e2778dad3 in ?? ()
18 LWP 7897 "rpc worker-7897" 0x00007f8e2778dad3 in ?? ()
19 LWP 7898 "rpc worker-7898" 0x00007f8e2778dad3 in ?? ()
20 LWP 7899 "rpc worker-7899" 0x00007f8e2778dad3 in ?? ()
21 LWP 7900 "rpc worker-7900" 0x00007f8e2778dad3 in ?? ()
22 LWP 7901 "rpc worker-7901" 0x00007f8e2778dad3 in ?? ()
23 LWP 7902 "rpc worker-7902" 0x00007f8e2778dad3 in ?? ()
24 LWP 7903 "rpc worker-7903" 0x00007f8e2778dad3 in ?? ()
25 LWP 7904 "rpc worker-7904" 0x00007f8e2778dad3 in ?? ()
26 LWP 7905 "rpc worker-7905" 0x00007f8e2778dad3 in ?? ()
27 LWP 7906 "rpc worker-7906" 0x00007f8e2778dad3 in ?? ()
28 LWP 7907 "rpc worker-7907" 0x00007f8e2778dad3 in ?? ()
29 LWP 7908 "rpc worker-7908" 0x00007f8e2778dad3 in ?? ()
30 LWP 7909 "rpc worker-7909" 0x00007f8e2778dad3 in ?? ()
31 LWP 7910 "rpc worker-7910" 0x00007f8e2778dad3 in ?? ()
32 LWP 7911 "rpc worker-7911" 0x00007f8e2778dad3 in ?? ()
33 LWP 7912 "rpc worker-7912" 0x00007f8e2778dad3 in ?? ()
34 LWP 7913 "rpc worker-7913" 0x00007f8e2778dad3 in ?? ()
35 LWP 7914 "rpc worker-7914" 0x00007f8e2778dad3 in ?? ()
36 LWP 7915 "rpc worker-7915" 0x00007f8e2778dad3 in ?? ()
37 LWP 7916 "rpc worker-7916" 0x00007f8e2778dad3 in ?? ()
38 LWP 7917 "rpc worker-7917" 0x00007f8e2778dad3 in ?? ()
39 LWP 7918 "rpc worker-7918" 0x00007f8e2778dad3 in ?? ()
40 LWP 7919 "rpc worker-7919" 0x00007f8e2778dad3 in ?? ()
41 LWP 7920 "rpc worker-7920" 0x00007f8e2778dad3 in ?? ()
42 LWP 7921 "rpc worker-7921" 0x00007f8e2778dad3 in ?? ()
43 LWP 7922 "rpc worker-7922" 0x00007f8e2778dad3 in ?? ()
44 LWP 7923 "rpc worker-7923" 0x00007f8e2778dad3 in ?? ()
45 LWP 7924 "rpc worker-7924" 0x00007f8e2778dad3 in ?? ()
46 LWP 7925 "rpc worker-7925" 0x00007f8e2778dad3 in ?? ()
47 LWP 7926 "rpc worker-7926" 0x00007f8e2778dad3 in ?? ()
48 LWP 7927 "rpc worker-7927" 0x00007f8e2778dad3 in ?? ()
49 LWP 7928 "rpc worker-7928" 0x00007f8e2778dad3 in ?? ()
50 LWP 7929 "rpc worker-7929" 0x00007f8e2778dad3 in ?? ()
51 LWP 7930 "rpc worker-7930" 0x00007f8e2778dad3 in ?? ()
52 LWP 7931 "rpc worker-7931" 0x00007f8e2778dad3 in ?? ()
53 LWP 7932 "rpc worker-7932" 0x00007f8e2778dad3 in ?? ()
54 LWP 7933 "rpc worker-7933" 0x00007f8e2778dad3 in ?? ()
55 LWP 7934 "rpc worker-7934" 0x00007f8e2778dad3 in ?? ()
56 LWP 7935 "rpc worker-7935" 0x00007f8e2778dad3 in ?? ()
57 LWP 7936 "rpc worker-7936" 0x00007f8e2778dad3 in ?? ()
58 LWP 7937 "rpc worker-7937" 0x00007f8e2778dad3 in ?? ()
59 LWP 7938 "rpc worker-7938" 0x00007f8e2778dad3 in ?? ()
60 LWP 7939 "rpc worker-7939" 0x00007f8e2778dad3 in ?? ()
61 LWP 7940 "rpc worker-7940" 0x00007f8e2778dad3 in ?? ()
62 LWP 7941 "rpc worker-7941" 0x00007f8e2778dad3 in ?? ()
63 LWP 7942 "rpc worker-7942" 0x00007f8e2778dad3 in ?? ()
64 LWP 7943 "rpc worker-7943" 0x00007f8e2778dad3 in ?? ()
65 LWP 7944 "rpc worker-7944" 0x00007f8e2778dad3 in ?? ()
66 LWP 7945 "rpc worker-7945" 0x00007f8e2778dad3 in ?? ()
67 LWP 7946 "rpc worker-7946" 0x00007f8e2778dad3 in ?? ()
68 LWP 7947 "rpc worker-7947" 0x00007f8e2778dad3 in ?? ()
69 LWP 7948 "rpc worker-7948" 0x00007f8e2778dad3 in ?? ()
70 LWP 7949 "rpc worker-7949" 0x00007f8e2778dad3 in ?? ()
71 LWP 7950 "rpc worker-7950" 0x00007f8e2778dad3 in ?? ()
72 LWP 7951 "rpc worker-7951" 0x00007f8e2778dad3 in ?? ()
73 LWP 7952 "rpc worker-7952" 0x00007f8e2778dad3 in ?? ()
74 LWP 7953 "rpc worker-7953" 0x00007f8e2778dad3 in ?? ()
75 LWP 7954 "rpc worker-7954" 0x00007f8e2778dad3 in ?? ()
76 LWP 7955 "rpc worker-7955" 0x00007f8e2778dad3 in ?? ()
77 LWP 7956 "rpc worker-7956" 0x00007f8e2778dad3 in ?? ()
78 LWP 7957 "rpc worker-7957" 0x00007f8e2778dad3 in ?? ()
79 LWP 7958 "rpc worker-7958" 0x00007f8e2778dad3 in ?? ()
80 LWP 7959 "rpc worker-7959" 0x00007f8e2778dad3 in ?? ()
81 LWP 7960 "rpc worker-7960" 0x00007f8e2778dad3 in ?? ()
82 LWP 7961 "rpc worker-7961" 0x00007f8e2778dad3 in ?? ()
83 LWP 7962 "rpc worker-7962" 0x00007f8e2778dad3 in ?? ()
84 LWP 7963 "rpc worker-7963" 0x00007f8e2778dad3 in ?? ()
85 LWP 7964 "rpc worker-7964" 0x00007f8e2778dad3 in ?? ()
86 LWP 7965 "rpc worker-7965" 0x00007f8e2778dad3 in ?? ()
87 LWP 7966 "rpc worker-7966" 0x00007f8e2778dad3 in ?? ()
88 LWP 7967 "rpc worker-7967" 0x00007f8e2778dad3 in ?? ()
89 LWP 7968 "rpc worker-7968" 0x00007f8e2778dad3 in ?? ()
90 LWP 7969 "rpc worker-7969" 0x00007f8e2778dad3 in ?? ()
91 LWP 7970 "rpc worker-7970" 0x00007f8e2778dad3 in ?? ()
92 LWP 7971 "rpc worker-7971" 0x00007f8e2778dad3 in ?? ()
93 LWP 7972 "rpc worker-7972" 0x00007f8e2778dad3 in ?? ()
94 LWP 7973 "rpc worker-7973" 0x00007f8e2778dad3 in ?? ()
95 LWP 7974 "rpc worker-7974" 0x00007f8e2778dad3 in ?? ()
96 LWP 7975 "rpc worker-7975" 0x00007f8e2778dad3 in ?? ()
97 LWP 7976 "rpc worker-7976" 0x00007f8e2778dad3 in ?? ()
98 LWP 7977 "rpc worker-7977" 0x00007f8e2778dad3 in ?? ()
99 LWP 7978 "rpc worker-7978" 0x00007f8e2778dad3 in ?? ()
100 LWP 7979 "rpc worker-7979" 0x00007f8e2778dad3 in ?? ()
101 LWP 7980 "rpc worker-7980" 0x00007f8e2778dad3 in ?? ()
102 LWP 7981 "rpc worker-7981" 0x00007f8e2778dad3 in ?? ()
103 LWP 7982 "rpc worker-7982" 0x00007f8e2778dad3 in ?? ()
104 LWP 7983 "rpc worker-7983" 0x00007f8e2778dad3 in ?? ()
105 LWP 7984 "rpc worker-7984" 0x00007f8e2778dad3 in ?? ()
106 LWP 7985 "rpc worker-7985" 0x00007f8e2778dad3 in ?? ()
107 LWP 7986 "rpc worker-7986" 0x00007f8e2778dad3 in ?? ()
108 LWP 7987 "rpc worker-7987" 0x00007f8e2778dad3 in ?? ()
109 LWP 7988 "rpc worker-7988" 0x00007f8e2778dad3 in ?? ()
110 LWP 7989 "rpc worker-7989" 0x00007f8e2778dad3 in ?? ()
111 LWP 7990 "rpc worker-7990" 0x00007f8e2778dad3 in ?? ()
112 LWP 7991 "rpc worker-7991" 0x00007f8e2778dad3 in ?? ()
113 LWP 7992 "rpc worker-7992" 0x00007f8e2778dad3 in ?? ()
114 LWP 7993 "rpc worker-7993" 0x00007f8e2778dad3 in ?? ()
115 LWP 7994 "rpc worker-7994" 0x00007f8e2778dad3 in ?? ()
116 LWP 7995 "rpc worker-7995" 0x00007f8e2778dad3 in ?? ()
117 LWP 7996 "diag-logger-799" 0x00007f8e2778dfb9 in ?? ()
118 LWP 7997 "result-tracker-" 0x00007f8e2778dfb9 in ?? ()
119 LWP 7998 "excess-log-dele" 0x00007f8e2778dfb9 in ?? ()
120 LWP 7999 "acceptor-7999" 0x00007f8e205000c7 in ?? ()
121 LWP 8000 "heartbeat-8000" 0x00007f8e2778dfb9 in ?? ()
122 LWP 8001 "maintenance_sch" 0x00007f8e2778dfb9 in ?? ()
123 LWP 8240 "raft [worker]-8" 0x00007f8e2778dfb9 in ?? ()
124 LWP 8267 "wal-append [wor" 0x00007f8e2778dfb9 in ?? ()
Thread 124 (LWP 8267):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x0000602000188cf8 in ?? ()
#2 0x0000000000000001 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x000061a0001501b4 in ?? ()
#5 0x00007f8ddc038090 in ?? ()
#6 0x0000000000000003 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f8ddc0380b0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 123 (LWP 8240):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000032 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00007f8dd4dedb20 in ?? ()
#5 0x00007f8dd4ded7b0 in ?? ()
#6 0x0000000000000064 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 122 (LWP 8001):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000015 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f8ddc850550 in ?? ()
#6 0x000000000000002a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 8000):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x48e3b5d9e7beadbb in ?? ()
#2 0x000000000000000b in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007f8ddd0685b0 in ?? ()
#6 0x0000000000000017 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f8ddd0685d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f8ddd068650 in ?? ()
#12 0x00000ff1bba0d0cc in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 7999):
#0 0x00007f8e205000c7 in ?? ()
#1 0x00006140000222a8 in ?? ()
#2 0x00007f8ddd88fd50 in ?? ()
#3 0x00007f8ddd88fd80 in ?? ()
#4 0x00007f8ddd88fe80 in ?? ()
#5 0x00007f8ddd88fd70 in ?? ()
#6 0x00007f8ddd88fde0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007f8ddd88ff00 in ?? ()
#11 0x00007f8ddd88f63c in ?? ()
#12 0x00000032dd88f5b0 in ?? ()
#13 0x00007f8ddd093000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 7998):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x00007f8dde0a9f00 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 7997):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x00007f8dde8c20c0 in ?? ()
#2 0x0000000000000005 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000879b0 in ?? ()
#5 0x00007f8dde8c20b0 in ?? ()
#6 0x000000000000000a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 7996):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 7995):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 115 (LWP 7994):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 114 (LWP 7993):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 7992):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 7991):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 7990):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 7989):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 7988):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 7987):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 7986):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 7985):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 7984):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 7983):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 7982):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 7981):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 7980):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 7979):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 7978):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 7977):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 7976):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 7975):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 7974):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 7973):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 7972):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 7971):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 7970):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 7969):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 7968):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 7967):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 7966):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 7965):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 7964):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 7963):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 7962):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 7961):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 7960):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 7959):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 7958):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 7957):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 7956):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 7955):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x00000000000000ca in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007f8df3cb2e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8df3cb2e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 7954):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000004 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001a6888 in ?? ()
#4 0x00007f8df44cae50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8df44cae70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 74 (LWP 7953):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000063 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a008c in ?? ()
#4 0x00007f8df4ce2e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8df4ce2e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a0040 in ?? ()
#9 0x00007f8e2778d770 in ?? ()
#10 0x00007f8df4ce2e70 in ?? ()
#11 0x00000ff1be99c5d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 73 (LWP 7952):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x00000000000000bd in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001967cc in ?? ()
#4 0x00007f8df54fae50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8df54fae70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d000196780 in ?? ()
#9 0x00007f8e2778d770 in ?? ()
#10 0x00007f8df54fae70 in ?? ()
#11 0x00000ff1bea9f5d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 72 (LWP 7951):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x00000000000000c9 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00018ffcc in ?? ()
#4 0x00007f8df5d12e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8df5d12e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00018ff80 in ?? ()
#9 0x00007f8e2778d770 in ?? ()
#10 0x00007f8df5d12e70 in ?? ()
#11 0x00000ff1beba25d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 71 (LWP 7950):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 7949):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 7948):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 7947):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 7946):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 7945):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 7944):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 7943):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 7942):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 7941):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 7940):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 7939):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 7938):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 7937):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 7936):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 7935):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007f8dfde93e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8dfde93e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007f8e2778d770 in ?? ()
#10 0x00007f8dfde93e70 in ?? ()
#11 0x00000ff1bfbd27d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 7934):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012003c in ?? ()
#4 0x00007f8dfe6abe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8dfe6abe70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011fff0 in ?? ()
#9 0x00007f8e2778d770 in ?? ()
#10 0x00007f8dfe6abe70 in ?? ()
#11 0x00000ff1bfcd57d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 54 (LWP 7933):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00011d048 in ?? ()
#4 0x00007f8dfeec3e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8dfeec3e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 53 (LWP 7932):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000116848 in ?? ()
#4 0x00007f8dff6dbe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8dff6dbe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 52 (LWP 7931):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 7930):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 7929):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 7928):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 7927):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 7926):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 7925):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 7924):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 7923):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 7922):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 7921):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 7920):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 7919):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 7918):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 7917):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 7916):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 7915):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000071 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f8e08074e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8e08074e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f8e2778d770 in ?? ()
#10 0x00007f8e08074e70 in ?? ()
#11 0x00000ff1c100e9d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 7914):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000053 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f8e0888ce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8e0888ce70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f8e2778d770 in ?? ()
#10 0x00007f8e0888ce70 in ?? ()
#11 0x00000ff1c11119d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 34 (LWP 7913):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000005 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000967fc in ?? ()
#4 0x00007f8e090a4e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8e090a4e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000967b0 in ?? ()
#9 0x00007f8e2778d770 in ?? ()
#10 0x00007f8e090a4e70 in ?? ()
#11 0x00000ff1c12149d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 7912):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000018 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008fff8 in ?? ()
#4 0x00007f8e098bce50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f8e098bce70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 32 (LWP 7911):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 7910):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 7909):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 7908):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 7907):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 7906):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 7905):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 7904):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 7903):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 7902):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 7901):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 7900):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 7899):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 7898):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 7897):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 7896):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 7895):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x00007f8e12287c80 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 15 (LWP 7894):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 7893):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x00007f8e132af1c0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 13 (LWP 7892):
#0 0x00007f8e2778dad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 7891):
#0 0x00007f8e204fea47 in ?? ()
#1 0x00007f8e142b22e0 in ?? ()
#2 0x000061a00000f680 in ?? ()
#3 0x00007f8e142b22d0 in ?? ()
#4 0x00007f8e142b24e0 in ?? ()
#5 0x00007f8e142b2320 in ?? ()
#6 0x0000614000022498 in ?? ()
#7 0x00007f8e142b23a0 in ?? ()
#8 0x00007f8e22b5e25d in ?? ()
#9 0x3fb9773732db8000 in ?? ()
#10 0x000061a00000f680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f680 in ?? ()
#13 0x00000000a2000000 in ?? ()
#14 0x00007f8e00000000 in ?? ()
#15 0x41d9e1e0d82ea204 in ?? ()
#16 0x00000ff24284e474 in ?? ()
#17 0x00007f8e142b2380 in ?? ()
#18 0x00007f8e22b62ba3 in ?? ()
#19 0x00007f8e142b2350 in ?? ()
#20 0x3fb9773732db8000 in ?? ()
#21 0x00000000142b23a0 in ?? ()
#22 0x000061a00000f680 in ?? ()
#23 0x0000614000022498 in ?? ()
#24 0x3fb9773732db8000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 7890):
#0 0x00007f8e204fea47 in ?? ()
#1 0x00007f8e14ab32e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f8e14ab32d0 in ?? ()
#4 0x00007f8e14ab34e0 in ?? ()
#5 0x00007f8e14ab3320 in ?? ()
#6 0x0000614000022298 in ?? ()
#7 0x00007f8e14ab33a0 in ?? ()
#8 0x00007f8e22b5e25d in ?? ()
#9 0x3fb984f2bae50000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x0000000088000000 in ?? ()
#14 0x00007f8e00000000 in ?? ()
#15 0x41d9e1e0d82ea202 in ?? ()
#16 0x00000ff24294e674 in ?? ()
#17 0x00007f8e14ab3380 in ?? ()
#18 0x00007f8e22b62ba3 in ?? ()
#19 0x00007f8e14ab3350 in ?? ()
#20 0x3fb984f2bae50000 in ?? ()
#21 0x0000000014ab33a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000022298 in ?? ()
#24 0x3fb984f2bae50000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 7889):
#0 0x00007f8e204fea47 in ?? ()
#1 0x00007f8e152b42e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f8e152b42d0 in ?? ()
#4 0x00007f8e152b44e0 in ?? ()
#5 0x00007f8e152b4320 in ?? ()
#6 0x0000614000022098 in ?? ()
#7 0x00007f8e152b43a0 in ?? ()
#8 0x00007f8e22b5e25d in ?? ()
#9 0x3fb97fcbabfa0000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x00000000b3800000 in ?? ()
#14 0x00007f8e00000000 in ?? ()
#15 0x41d9e1e0d82ea205 in ?? ()
#16 0x00000ff242a4e874 in ?? ()
#17 0x00007f8e152b4380 in ?? ()
#18 0x00007f8e22b62ba3 in ?? ()
#19 0x00007f8e152b4350 in ?? ()
#20 0x3fb97fcbabfa0000 in ?? ()
#21 0x00000000152b43a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000022098 in ?? ()
#24 0x3fb97fcbabfa0000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 7888):
#0 0x00007f8e204fea47 in ?? ()
#1 0x00007f8e172ad2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f8e172ad2d0 in ?? ()
#4 0x00007f8e172ad4e0 in ?? ()
#5 0x00007f8e172ad320 in ?? ()
#6 0x0000614000021e98 in ?? ()
#7 0x00007f8e172ad3a0 in ?? ()
#8 0x00007f8e22b5e25d in ?? ()
#9 0x3fb9747d37558000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x00000000ddc00000 in ?? ()
#14 0x00007f8e00000000 in ?? ()
#15 0x41d9e1e0d82ea208 in ?? ()
#16 0x00000ff242e4da74 in ?? ()
#17 0x00007f8e172ad380 in ?? ()
#18 0x00007f8e22b62ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 7885):
#0 0x00007f8e204f1cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f8e18cb7788 in ?? ()
#3 0x0000602000018790 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 7884):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x00007f8e184b6080 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 7883):
#0 0x00007f8e277919e2 in ?? ()
#1 0x00007f8e17cb3b60 in ?? ()
#2 0x00007f8e17cb3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f8e17cb3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f8e17cb3c10 in ?? ()
#8 0x0000611000088950 in ?? ()
#9 0x00007f8e17cb3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f8e2d0b5f0c in ?? ()
#12 0x00007f8e2d0a57b9 in ?? ()
#13 0x00007f8e2d0a9da6 in ?? ()
#14 0x00007f8e2d0ae7e0 in ?? ()
#15 0x00007f8e2d0ae609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f8e243a598e in ?? ()
#18 0x00007f8e277876db in ?? ()
#19 0x00000ff242f8e67c in ?? ()
#20 0x00007f8e17cb3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 7877):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x00007f8e19cb8c40 in ?? ()
#2 0x000000000000001a in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f8e19cb8c30 in ?? ()
#6 0x0000000000000034 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 7876):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 7875):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 7874):
#0 0x00007f8e2778dfb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f8e1b4bc410 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 7873):
#0 0x00007f8e27791d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250115 10:02:17.581073 7644 external_mini_cluster-itest-base.cc:86] Attempting to dump stacks of TS 2 with UUID ad9d5d231fad4f0290f5d987cea32078 and pid 8003
************************ BEGIN STACKS **************************
[New LWP 8004]
[New LWP 8005]
[New LWP 8006]
[New LWP 8007]
[New LWP 8013]
[New LWP 8014]
[New LWP 8015]
[New LWP 8018]
[New LWP 8019]
[New LWP 8020]
[New LWP 8021]
[New LWP 8022]
[New LWP 8023]
[New LWP 8024]
[New LWP 8025]
[New LWP 8026]
[New LWP 8027]
[New LWP 8028]
[New LWP 8029]
[New LWP 8030]
[New LWP 8031]
[New LWP 8032]
[New LWP 8033]
[New LWP 8034]
[New LWP 8035]
[New LWP 8036]
[New LWP 8037]
[New LWP 8038]
[New LWP 8039]
[New LWP 8040]
[New LWP 8041]
[New LWP 8042]
[New LWP 8043]
[New LWP 8044]
[New LWP 8045]
[New LWP 8046]
[New LWP 8047]
[New LWP 8048]
[New LWP 8049]
[New LWP 8050]
[New LWP 8051]
[New LWP 8052]
[New LWP 8053]
[New LWP 8054]
[New LWP 8055]
[New LWP 8056]
[New LWP 8057]
[New LWP 8058]
[New LWP 8059]
[New LWP 8060]
[New LWP 8061]
[New LWP 8062]
[New LWP 8063]
[New LWP 8064]
[New LWP 8065]
[New LWP 8066]
[New LWP 8067]
[New LWP 8068]
[New LWP 8069]
[New LWP 8070]
[New LWP 8071]
[New LWP 8072]
[New LWP 8073]
[New LWP 8074]
[New LWP 8075]
[New LWP 8076]
[New LWP 8077]
[New LWP 8078]
[New LWP 8079]
[New LWP 8080]
[New LWP 8081]
[New LWP 8082]
[New LWP 8083]
[New LWP 8084]
[New LWP 8085]
[New LWP 8086]
[New LWP 8087]
[New LWP 8088]
[New LWP 8089]
[New LWP 8090]
[New LWP 8091]
[New LWP 8092]
[New LWP 8093]
[New LWP 8094]
[New LWP 8095]
[New LWP 8096]
[New LWP 8097]
[New LWP 8098]
[New LWP 8099]
[New LWP 8100]
[New LWP 8101]
[New LWP 8102]
[New LWP 8103]
[New LWP 8104]
[New LWP 8105]
[New LWP 8106]
[New LWP 8107]
[New LWP 8108]
[New LWP 8109]
[New LWP 8110]
[New LWP 8111]
[New LWP 8112]
[New LWP 8113]
[New LWP 8114]
[New LWP 8115]
[New LWP 8116]
[New LWP 8117]
[New LWP 8118]
[New LWP 8119]
[New LWP 8120]
[New LWP 8121]
[New LWP 8122]
[New LWP 8123]
[New LWP 8124]
[New LWP 8125]
[New LWP 8126]
[New LWP 8127]
[New LWP 8128]
[New LWP 8129]
[New LWP 8130]
[New LWP 8131]
[New LWP 8278]
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b51
Cannot access memory at address 0x5f35535f35534b49
0x00007f1599825d50 in ?? ()
Id Target Id Frame
* 1 LWP 8003 "kudu" 0x00007f1599825d50 in ?? ()
2 LWP 8004 "kudu" 0x00007f1599821fb9 in ?? ()
3 LWP 8005 "kudu" 0x00007f1599821fb9 in ?? ()
4 LWP 8006 "kudu" 0x00007f1599821fb9 in ?? ()
5 LWP 8007 "kernel-watcher-" 0x00007f1599821fb9 in ?? ()
6 LWP 8013 "ntp client-8013" 0x00007f15998259e2 in ?? ()
7 LWP 8014 "file cache-evic" 0x00007f1599821fb9 in ?? ()
8 LWP 8015 "sq_acceptor" 0x00007f1592585cb9 in ?? ()
9 LWP 8018 "rpc reactor-801" 0x00007f1592592a47 in ?? ()
10 LWP 8019 "rpc reactor-801" 0x00007f1592592a47 in ?? ()
11 LWP 8020 "rpc reactor-802" 0x00007f1592592a47 in ?? ()
12 LWP 8021 "rpc reactor-802" 0x00007f1592592a47 in ?? ()
13 LWP 8022 "MaintenanceMgr " 0x00007f1599821ad3 in ?? ()
14 LWP 8023 "txn-status-mana" 0x00007f1599821fb9 in ?? ()
15 LWP 8024 "collect_and_rem" 0x00007f1599821fb9 in ?? ()
16 LWP 8025 "tc-session-exp-" 0x00007f1599821fb9 in ?? ()
17 LWP 8026 "rpc worker-8026" 0x00007f1599821ad3 in ?? ()
18 LWP 8027 "rpc worker-8027" 0x00007f1599821ad3 in ?? ()
19 LWP 8028 "rpc worker-8028" 0x00007f1599821ad3 in ?? ()
20 LWP 8029 "rpc worker-8029" 0x00007f1599821ad3 in ?? ()
21 LWP 8030 "rpc worker-8030" 0x00007f1599821ad3 in ?? ()
22 LWP 8031 "rpc worker-8031" 0x00007f1599821ad3 in ?? ()
23 LWP 8032 "rpc worker-8032" 0x00007f1599821ad3 in ?? ()
24 LWP 8033 "rpc worker-8033" 0x00007f1599821ad3 in ?? ()
25 LWP 8034 "rpc worker-8034" 0x00007f1599821ad3 in ?? ()
26 LWP 8035 "rpc worker-8035" 0x00007f1599821ad3 in ?? ()
27 LWP 8036 "rpc worker-8036" 0x00007f1599821ad3 in ?? ()
28 LWP 8037 "rpc worker-8037" 0x00007f1599821ad3 in ?? ()
29 LWP 8038 "rpc worker-8038" 0x00007f1599821ad3 in ?? ()
30 LWP 8039 "rpc worker-8039" 0x00007f1599821ad3 in ?? ()
31 LWP 8040 "rpc worker-8040" 0x00007f1599821ad3 in ?? ()
32 LWP 8041 "rpc worker-8041" 0x00007f1599821ad3 in ?? ()
33 LWP 8042 "rpc worker-8042" 0x00007f1599821ad3 in ?? ()
34 LWP 8043 "rpc worker-8043" 0x00007f1599821ad3 in ?? ()
35 LWP 8044 "rpc worker-8044" 0x00007f1599821ad3 in ?? ()
36 LWP 8045 "rpc worker-8045" 0x00007f1599821ad3 in ?? ()
37 LWP 8046 "rpc worker-8046" 0x00007f1599821ad3 in ?? ()
38 LWP 8047 "rpc worker-8047" 0x00007f1599821ad3 in ?? ()
39 LWP 8048 "rpc worker-8048" 0x00007f1599821ad3 in ?? ()
40 LWP 8049 "rpc worker-8049" 0x00007f1599821ad3 in ?? ()
41 LWP 8050 "rpc worker-8050" 0x00007f1599821ad3 in ?? ()
42 LWP 8051 "rpc worker-8051" 0x00007f1599821ad3 in ?? ()
43 LWP 8052 "rpc worker-8052" 0x00007f1599821ad3 in ?? ()
44 LWP 8053 "rpc worker-8053" 0x00007f1599821ad3 in ?? ()
45 LWP 8054 "rpc worker-8054" 0x00007f1599821ad3 in ?? ()
46 LWP 8055 "rpc worker-8055" 0x00007f1599821ad3 in ?? ()
47 LWP 8056 "rpc worker-8056" 0x00007f1599821ad3 in ?? ()
48 LWP 8057 "rpc worker-8057" 0x00007f1599821ad3 in ?? ()
49 LWP 8058 "rpc worker-8058" 0x00007f1599821ad3 in ?? ()
50 LWP 8059 "rpc worker-8059" 0x00007f1599821ad3 in ?? ()
51 LWP 8060 "rpc worker-8060" 0x00007f1599821ad3 in ?? ()
52 LWP 8061 "rpc worker-8061" 0x00007f1599821ad3 in ?? ()
53 LWP 8062 "rpc worker-8062" 0x00007f1599821ad3 in ?? ()
54 LWP 8063 "rpc worker-8063" 0x00007f1599821ad3 in ?? ()
55 LWP 8064 "rpc worker-8064" 0x00007f1599821ad3 in ?? ()
56 LWP 8065 "rpc worker-8065" 0x00007f1599821ad3 in ?? ()
57 LWP 8066 "rpc worker-8066" 0x00007f1599821ad3 in ?? ()
58 LWP 8067 "rpc worker-8067" 0x00007f1599821ad3 in ?? ()
59 LWP 8068 "rpc worker-8068" 0x00007f1599821ad3 in ?? ()
60 LWP 8069 "rpc worker-8069" 0x00007f1599821ad3 in ?? ()
61 LWP 8070 "rpc worker-8070" 0x00007f1599821ad3 in ?? ()
62 LWP 8071 "rpc worker-8071" 0x00007f1599821ad3 in ?? ()
63 LWP 8072 "rpc worker-8072" 0x00007f1599821ad3 in ?? ()
64 LWP 8073 "rpc worker-8073" 0x00007f1599821ad3 in ?? ()
65 LWP 8074 "rpc worker-8074" 0x00007f1599821ad3 in ?? ()
66 LWP 8075 "rpc worker-8075" 0x00007f1599821ad3 in ?? ()
67 LWP 8076 "rpc worker-8076" 0x00007f1599821ad3 in ?? ()
68 LWP 8077 "rpc worker-8077" 0x00007f1599821ad3 in ?? ()
69 LWP 8078 "rpc worker-8078" 0x00007f1599821ad3 in ?? ()
70 LWP 8079 "rpc worker-8079" 0x00007f1599821ad3 in ?? ()
71 LWP 8080 "rpc worker-8080" 0x00007f1599821ad3 in ?? ()
72 LWP 8081 "rpc worker-8081" 0x00007f1599821ad3 in ?? ()
73 LWP 8082 "rpc worker-8082" 0x00007f1599821ad3 in ?? ()
74 LWP 8083 "rpc worker-8083" 0x00007f1599821ad3 in ?? ()
75 LWP 8084 "rpc worker-8084" 0x00007f1599821ad3 in ?? ()
76 LWP 8085 "rpc worker-8085" 0x00007f1599821ad3 in ?? ()
77 LWP 8086 "rpc worker-8086" 0x00007f1599821ad3 in ?? ()
78 LWP 8087 "rpc worker-8087" 0x00007f1599821ad3 in ?? ()
79 LWP 8088 "rpc worker-8088" 0x00007f1599821ad3 in ?? ()
80 LWP 8089 "rpc worker-8089" 0x00007f1599821ad3 in ?? ()
81 LWP 8090 "rpc worker-8090" 0x00007f1599821ad3 in ?? ()
82 LWP 8091 "rpc worker-8091" 0x00007f1599821ad3 in ?? ()
83 LWP 8092 "rpc worker-8092" 0x00007f1599821ad3 in ?? ()
84 LWP 8093 "rpc worker-8093" 0x00007f1599821ad3 in ?? ()
85 LWP 8094 "rpc worker-8094" 0x00007f1599821ad3 in ?? ()
86 LWP 8095 "rpc worker-8095" 0x00007f1599821ad3 in ?? ()
87 LWP 8096 "rpc worker-8096" 0x00007f1599821ad3 in ?? ()
88 LWP 8097 "rpc worker-8097" 0x00007f1599821ad3 in ?? ()
89 LWP 8098 "rpc worker-8098" 0x00007f1599821ad3 in ?? ()
90 LWP 8099 "rpc worker-8099" 0x00007f1599821ad3 in ?? ()
91 LWP 8100 "rpc worker-8100" 0x00007f1599821ad3 in ?? ()
92 LWP 8101 "rpc worker-8101" 0x00007f1599821ad3 in ?? ()
93 LWP 8102 "rpc worker-8102" 0x00007f1599821ad3 in ?? ()
94 LWP 8103 "rpc worker-8103" 0x00007f1599821ad3 in ?? ()
95 LWP 8104 "rpc worker-8104" 0x00007f1599821ad3 in ?? ()
96 LWP 8105 "rpc worker-8105" 0x00007f1599821ad3 in ?? ()
97 LWP 8106 "rpc worker-8106" 0x00007f1599821ad3 in ?? ()
98 LWP 8107 "rpc worker-8107" 0x00007f1599821ad3 in ?? ()
99 LWP 8108 "rpc worker-8108" 0x00007f1599821ad3 in ?? ()
100 LWP 8109 "rpc worker-8109" 0x00007f1599821ad3 in ?? ()
101 LWP 8110 "rpc worker-8110" 0x00007f1599821ad3 in ?? ()
102 LWP 8111 "rpc worker-8111" 0x00007f1599821ad3 in ?? ()
103 LWP 8112 "rpc worker-8112" 0x00007f1599821ad3 in ?? ()
104 LWP 8113 "rpc worker-8113" 0x00007f1599821ad3 in ?? ()
105 LWP 8114 "rpc worker-8114" 0x00007f1599821ad3 in ?? ()
106 LWP 8115 "rpc worker-8115" 0x00007f1599821ad3 in ?? ()
107 LWP 8116 "rpc worker-8116" 0x00007f1599821ad3 in ?? ()
108 LWP 8117 "rpc worker-8117" 0x00007f1599821ad3 in ?? ()
109 LWP 8118 "rpc worker-8118" 0x00007f1599821ad3 in ?? ()
110 LWP 8119 "rpc worker-8119" 0x00007f1599821ad3 in ?? ()
111 LWP 8120 "rpc worker-8120" 0x00007f1599821ad3 in ?? ()
112 LWP 8121 "rpc worker-8121" 0x00007f1599821ad3 in ?? ()
113 LWP 8122 "rpc worker-8122" 0x00007f1599821ad3 in ?? ()
114 LWP 8123 "rpc worker-8123" 0x00007f1599821ad3 in ?? ()
115 LWP 8124 "rpc worker-8124" 0x00007f1599821ad3 in ?? ()
116 LWP 8125 "rpc worker-8125" 0x00007f1599821ad3 in ?? ()
117 LWP 8126 "diag-logger-812" 0x00007f1599821fb9 in ?? ()
118 LWP 8127 "result-tracker-" 0x00007f1599821fb9 in ?? ()
119 LWP 8128 "excess-log-dele" 0x00007f1599821fb9 in ?? ()
120 LWP 8129 "acceptor-8129" 0x00007f15925940c7 in ?? ()
121 LWP 8130 "heartbeat-8130" 0x00007f1599821fb9 in ?? ()
122 LWP 8131 "maintenance_sch" 0x00007f1599821fb9 in ?? ()
123 LWP 8278 "raft [worker]-8" 0x00007f1599821fb9 in ?? ()
Thread 123 (LWP 8278):
#0 0x00007f1599821fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000009 in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x00007f154bfdcb24 in ?? ()
#5 0x00007f154bfdc7b0 in ?? ()
#6 0x0000000000000013 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f154bfdc7d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x0000000000000000 in ?? ()
Thread 122 (LWP 8131):
#0 0x00007f1599821fb9 in ?? ()
#1 0x0000000045e0360e in ?? ()
#2 0x0000000000000015 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000616000017580 in ?? ()
#5 0x00007f154e8e4550 in ?? ()
#6 0x000000000000002a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 121 (LWP 8130):
#0 0x00007f1599821fb9 in ?? ()
#1 0x48e3b5d9e7beadbb in ?? ()
#2 0x000000000000000e in ?? ()
#3 0x0000000100000081 in ?? ()
#4 0x0000613000019404 in ?? ()
#5 0x00007f154f0fc5b0 in ?? ()
#6 0x000000000000001d in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x00007f154f0fc5d0 in ?? ()
#9 0x0000000200000000 in ?? ()
#10 0x0000008000000189 in ?? ()
#11 0x00007f154f0fc650 in ?? ()
#12 0x00000fe2a9e1f8cc in ?? ()
#13 0x0000000000000000 in ?? ()
Thread 120 (LWP 8129):
#0 0x00007f15925940c7 in ?? ()
#1 0x00006140000224a8 in ?? ()
#2 0x00007f154f923d50 in ?? ()
#3 0x00007f154f923d80 in ?? ()
#4 0x00007f154f923e80 in ?? ()
#5 0x00007f154f923d70 in ?? ()
#6 0x00007f154f923de0 in ?? ()
#7 0x0000000000000080 in ?? ()
#8 0x00000000008acceb in __sanitizer::theDepot ()
#9 0x0000000500000014 in ?? ()
#10 0x00007f154f923f00 in ?? ()
#11 0x00007f154f92363c in ?? ()
#12 0x000000324f9235b0 in ?? ()
#13 0x00007f154f127000 in ?? ()
#14 0x0000000000000000 in ?? ()
Thread 119 (LWP 8128):
#0 0x00007f1599821fb9 in ?? ()
#1 0x00007f155013df00 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 118 (LWP 8127):
#0 0x00007f1599821fb9 in ?? ()
#1 0x00007f15509560c0 in ?? ()
#2 0x0000000000000005 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x00006110000879b0 in ?? ()
#5 0x00007f15509560b0 in ?? ()
#6 0x000000000000000a in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 117 (LWP 8126):
#0 0x00007f1599821fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 116 (LWP 8125):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 115 (LWP 8124):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 114 (LWP 8123):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 113 (LWP 8122):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 112 (LWP 8121):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 111 (LWP 8120):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 110 (LWP 8119):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 109 (LWP 8118):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 108 (LWP 8117):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 107 (LWP 8116):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 106 (LWP 8115):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 105 (LWP 8114):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 104 (LWP 8113):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 103 (LWP 8112):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 102 (LWP 8111):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 101 (LWP 8110):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 100 (LWP 8109):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 99 (LWP 8108):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 98 (LWP 8107):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 97 (LWP 8106):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 96 (LWP 8105):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 95 (LWP 8104):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 94 (LWP 8103):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 93 (LWP 8102):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 92 (LWP 8101):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 91 (LWP 8100):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 90 (LWP 8099):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 89 (LWP 8098):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 88 (LWP 8097):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 87 (LWP 8096):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 86 (LWP 8095):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 85 (LWP 8094):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 84 (LWP 8093):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 83 (LWP 8092):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 82 (LWP 8091):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 81 (LWP 8090):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 80 (LWP 8089):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 79 (LWP 8088):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 78 (LWP 8087):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 77 (LWP 8086):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 76 (LWP 8085):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000106 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001ad088 in ?? ()
#4 0x00007f1565d46e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1565d46e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 75 (LWP 8084):
#0 0x00007f1599821ad3 in ?? ()
#1 0x000000000000002d in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a688c in ?? ()
#4 0x00007f156655ee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f156655ee70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a6840 in ?? ()
#9 0x00007f1599821770 in ?? ()
#10 0x00007f156655ee70 in ?? ()
#11 0x00000fe2accabdd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 74 (LWP 8083):
#0 0x00007f1599821ad3 in ?? ()
#1 0x00000000000000cf in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0001a008c in ?? ()
#4 0x00007f1566d76e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1566d76e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001a0040 in ?? ()
#9 0x00007f1599821770 in ?? ()
#10 0x00007f1566d76e70 in ?? ()
#11 0x00000fe2acdaedd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 73 (LWP 8082):
#0 0x00007f1599821ad3 in ?? ()
#1 0x00000000000000c0 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d0001967c8 in ?? ()
#4 0x00007f156758ee50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f156758ee70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 72 (LWP 8081):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 71 (LWP 8080):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 70 (LWP 8079):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 69 (LWP 8078):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 68 (LWP 8077):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 67 (LWP 8076):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 66 (LWP 8075):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 65 (LWP 8074):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 64 (LWP 8073):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 63 (LWP 8072):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 62 (LWP 8071):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 61 (LWP 8070):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 60 (LWP 8069):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 59 (LWP 8068):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 58 (LWP 8067):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 57 (LWP 8066):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 56 (LWP 8065):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00012683c in ?? ()
#4 0x00007f156ff27e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f156ff27e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0001267f0 in ?? ()
#9 0x00007f1599821770 in ?? ()
#10 0x00007f156ff27e70 in ?? ()
#11 0x00000fe2adfe4fd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 55 (LWP 8064):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000120038 in ?? ()
#4 0x00007f157073fe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f157073fe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 54 (LWP 8063):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000001 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00011d04c in ?? ()
#4 0x00007f1570f57e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f1570f57e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00011d000 in ?? ()
#9 0x00007f1599821770 in ?? ()
#10 0x00007f1570f57e70 in ?? ()
#11 0x00000fe2ae1eafd0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 53 (LWP 8062):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000002 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d000116848 in ?? ()
#4 0x00007f157176fe50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f157176fe70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 52 (LWP 8061):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 51 (LWP 8060):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 50 (LWP 8059):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 49 (LWP 8058):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 48 (LWP 8057):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 47 (LWP 8056):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 46 (LWP 8055):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 45 (LWP 8054):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 44 (LWP 8053):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 43 (LWP 8052):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 42 (LWP 8051):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 41 (LWP 8050):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 40 (LWP 8049):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 39 (LWP 8048):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 38 (LWP 8047):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 37 (LWP 8046):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 36 (LWP 8045):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000041 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000a67ec in ?? ()
#4 0x00007f157a108e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f157a108e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000a67a0 in ?? ()
#9 0x00007f1599821770 in ?? ()
#10 0x00007f157a108e70 in ?? ()
#11 0x00000fe2af4211d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 35 (LWP 8044):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000055 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d00009ffec in ?? ()
#4 0x00007f157a920e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f157a920e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d00009ffa0 in ?? ()
#9 0x00007f1599821770 in ?? ()
#10 0x00007f157a920e70 in ?? ()
#11 0x00000fe2af5241d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 34 (LWP 8043):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000015 in ?? ()
#2 0x0000000100000081 in ?? ()
#3 0x000060d0000967fc in ?? ()
#4 0x00007f157b138e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f157b138e70 in ?? ()
#7 0x0000000000000001 in ?? ()
#8 0x000060d0000967b0 in ?? ()
#9 0x00007f1599821770 in ?? ()
#10 0x00007f157b138e70 in ?? ()
#11 0x00000fe2af6271d0 in ?? ()
#12 0x0000000000000000 in ?? ()
Thread 33 (LWP 8042):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000034 in ?? ()
#2 0x0000000000000081 in ?? ()
#3 0x000060d00008fff8 in ?? ()
#4 0x00007f157b950e50 in ?? ()
#5 0x0000008000000000 in ?? ()
#6 0x00007f157b950e70 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 32 (LWP 8041):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 31 (LWP 8040):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 30 (LWP 8039):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 29 (LWP 8038):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 28 (LWP 8037):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 27 (LWP 8036):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 26 (LWP 8035):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 25 (LWP 8034):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 24 (LWP 8033):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 23 (LWP 8032):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 22 (LWP 8031):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 21 (LWP 8030):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 20 (LWP 8029):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 19 (LWP 8028):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 18 (LWP 8027):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 17 (LWP 8026):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 16 (LWP 8025):
#0 0x00007f1599821fb9 in ?? ()
#1 0x00007f1584317c80 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 15 (LWP 8024):
#0 0x00007f1599821fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 14 (LWP 8023):
#0 0x00007f1599821fb9 in ?? ()
#1 0x00007f15853551c0 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 13 (LWP 8022):
#0 0x00007f1599821ad3 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 12 (LWP 8021):
#0 0x00007f1592592a47 in ?? ()
#1 0x00007f15863942e0 in ?? ()
#2 0x000061a00000f680 in ?? ()
#3 0x00007f15863942d0 in ?? ()
#4 0x00007f15863944e0 in ?? ()
#5 0x00007f1586394320 in ?? ()
#6 0x0000614000022498 in ?? ()
#7 0x00007f15863943a0 in ?? ()
#8 0x00007f1594bf225d in ?? ()
#9 0x3fb3cba251094000 in ?? ()
#10 0x000061a00000f680 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f680 in ?? ()
#13 0x00000000f6000000 in ?? ()
#14 0x00007f1500000000 in ?? ()
#15 0x41d9e1e0d82ea201 in ?? ()
#16 0x00000fe330c6a874 in ?? ()
#17 0x00007f1586394380 in ?? ()
#18 0x00007f1594bf6ba3 in ?? ()
#19 0x00007f1586394350 in ?? ()
#20 0x3fb3cba251094000 in ?? ()
#21 0x00000000863943a0 in ?? ()
#22 0x000061a00000f680 in ?? ()
#23 0x0000614000022498 in ?? ()
#24 0x3fb3cba251094000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 11 (LWP 8020):
#0 0x00007f1592592a47 in ?? ()
#1 0x00007f1586bab2e0 in ?? ()
#2 0x000061a00000f080 in ?? ()
#3 0x00007f1586bab2d0 in ?? ()
#4 0x00007f1586bab4e0 in ?? ()
#5 0x00007f1586bab320 in ?? ()
#6 0x0000614000022298 in ?? ()
#7 0x00007f1586bab3a0 in ?? ()
#8 0x00007f1594bf225d in ?? ()
#9 0x3fb9596f7f5bc000 in ?? ()
#10 0x000061a00000f080 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000f080 in ?? ()
#13 0x0000000054800000 in ?? ()
#14 0x00007f1500000000 in ?? ()
#15 0x41d9e1e0d82ea205 in ?? ()
#16 0x00000fe330d6d674 in ?? ()
#17 0x00007f1586bab380 in ?? ()
#18 0x00007f1594bf6ba3 in ?? ()
#19 0x00007f1586bab350 in ?? ()
#20 0x3fb9596f7f5bc000 in ?? ()
#21 0x0000000086bab3a0 in ?? ()
#22 0x000061a00000f080 in ?? ()
#23 0x0000614000022298 in ?? ()
#24 0x3fb9596f7f5bc000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 10 (LWP 8019):
#0 0x00007f1592592a47 in ?? ()
#1 0x00007f15873b42e0 in ?? ()
#2 0x000061a00000ea80 in ?? ()
#3 0x00007f15873b42d0 in ?? ()
#4 0x00007f15873b44e0 in ?? ()
#5 0x00007f15873b4320 in ?? ()
#6 0x0000614000022098 in ?? ()
#7 0x00007f15873b43a0 in ?? ()
#8 0x00007f1594bf225d in ?? ()
#9 0x3fb9641d901d4000 in ?? ()
#10 0x000061a00000ea80 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000ea80 in ?? ()
#13 0x00000000d4000000 in ?? ()
#14 0x00007f1500000000 in ?? ()
#15 0x41d9e1e0d82ea205 in ?? ()
#16 0x00000fe330e6e874 in ?? ()
#17 0x00007f15873b4380 in ?? ()
#18 0x00007f1594bf6ba3 in ?? ()
#19 0x00007f15873b4350 in ?? ()
#20 0x3fb9641d901d4000 in ?? ()
#21 0x00000000873b43a0 in ?? ()
#22 0x000061a00000ea80 in ?? ()
#23 0x0000614000022098 in ?? ()
#24 0x3fb9641d901d4000 in ?? ()
#25 0x0000000000000000 in ?? ()
Thread 9 (LWP 8018):
#0 0x00007f1592592a47 in ?? ()
#1 0x00007f15893ad2e0 in ?? ()
#2 0x000061a00000e480 in ?? ()
#3 0x00007f15893ad2d0 in ?? ()
#4 0x00007f15893ad4e0 in ?? ()
#5 0x00007f15893ad320 in ?? ()
#6 0x0000614000021e98 in ?? ()
#7 0x00007f15893ad3a0 in ?? ()
#8 0x00007f1594bf225d in ?? ()
#9 0x3fb95b03cb7b0000 in ?? ()
#10 0x000061a00000e480 in ?? ()
#11 0x42a2309ce5400000 in ?? ()
#12 0x000061a00000e480 in ?? ()
#13 0x00000000a4000000 in ?? ()
#14 0x00007f1500000000 in ?? ()
#15 0x41d9e1e0d82ea204 in ?? ()
#16 0x00000fe33126da74 in ?? ()
#17 0x00007f15893ad380 in ?? ()
#18 0x00007f1594bf6ba3 in ?? ()
#19 0x0000000000000000 in ?? ()
Thread 8 (LWP 8015):
#0 0x00007f1592585cb9 in ?? ()
#1 0x00000000000000c8 in ?? ()
#2 0x00007f158adb7788 in ?? ()
#3 0x0000602000017790 in ?? ()
#4 0x0000000000000002 in ?? ()
#5 0x00000000000000c8 in ?? ()
#6 0x00000000008a4931 in __sanitizer::theDepot ()
#7 0x0000000000000000 in ?? ()
Thread 7 (LWP 8014):
#0 0x00007f1599821fb9 in ?? ()
#1 0x00007f158a5b6080 in ?? ()
#2 0x0000000000000000 in ?? ()
Thread 6 (LWP 8013):
#0 0x00007f15998259e2 in ?? ()
#1 0x00007f1589db3b60 in ?? ()
#2 0x00007f1589db3c00 in ?? ()
#3 0x000000000000001e in ?? ()
#4 0x0000000000000030 in ?? ()
#5 0x00007f1589db3bb0 in ?? ()
#6 0x000000000177bae0 in ?? ()
#7 0x00007f1589db3c10 in ?? ()
#8 0x0000611000088950 in ?? ()
#9 0x00007f1589db3c00 in ?? ()
#10 0x000000000089ee27 in __sanitizer::theDepot ()
#11 0x00007f159f149f0c in ?? ()
#12 0x00007f159f1397b9 in ?? ()
#13 0x00007f159f13dda6 in ?? ()
#14 0x00007f159f1427e0 in ?? ()
#15 0x00007f159f142609 in ?? ()
#16 0x0000000000a71ecd in __sanitizer::theDepot ()
#17 0x00007f159643998e in ?? ()
#18 0x00007f159981b6db in ?? ()
#19 0x00000fe3313ae67c in ?? ()
#20 0x00007f1589db3400 in ?? ()
#21 0x0000000000000000 in ?? ()
Thread 5 (LWP 8007):
#0 0x00007f1599821fb9 in ?? ()
#1 0x00007f158bdb8c40 in ?? ()
#2 0x000000000000001b in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x000061200000f728 in ?? ()
#5 0x00007f158bdb8c30 in ?? ()
#6 0x0000000000000036 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 4 (LWP 8006):
#0 0x00007f1599821fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 3 (LWP 8005):
#0 0x00007f1599821fb9 in ?? ()
#1 0x0000000000000000 in ?? ()
Thread 2 (LWP 8004):
#0 0x00007f1599821fb9 in ?? ()
#1 0x5f5347414c46000a in ?? ()
#2 0x0000000000000002 in ?? ()
#3 0x0000000000000081 in ?? ()
#4 0x0000612000016eb8 in ?? ()
#5 0x00007f158d5bc410 in ?? ()
#6 0x0000000000000004 in ?? ()
#7 0x0000000000000000 in ?? ()
Thread 1 (LWP 8003):
#0 0x00007f1599825d50 in ?? ()
#1 0x0000000000000000 in ?? ()
************************* END STACKS ***************************
I20250115 10:02:18.140236 7644 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu pid 7742
I20250115 10:02:18.151523 7864 generic_service.cc:196] Checking for leaks (request via RPC)
I20250115 10:02:18.284485 7644 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu with pid 7742
W20250115 10:02:18.312135 8021 connection.cc:537] server connection from 127.7.119.1:40193 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250115 10:02:18.312574 7644 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu pid 7873
I20250115 10:02:18.313640 7995 generic_service.cc:196] Checking for leaks (request via RPC)
I20250115 10:02:18.441962 7644 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu with pid 7873
I20250115 10:02:18.466755 7644 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu pid 8003
I20250115 10:02:18.473503 8125 generic_service.cc:196] Checking for leaks (request via RPC)
I20250115 10:02:18.602782 7644 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu with pid 8003
W20250115 10:02:18.627033 7667 connection.cc:537] server connection from 127.7.119.3:55035 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20250115 10:02:18.627311 7644 external_mini_cluster.cc:1661] Attempting to check leaks for /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu pid 7652
I20250115 10:02:18.628430 7713 generic_service.cc:196] Checking for leaks (request via RPC)
I20250115 10:02:18.735874 7644 external_mini_cluster.cc:1620] Killing /tmp/dist-test-taskeGzcyx/build/asan/bin/kudu with pid 7652
2025-01-15T10:02:18Z chronyd exiting
I20250115 10:02:18.771874 7644 test_util.cc:181] -----------------------------------------------
I20250115 10:02:18.771981 7644 test_util.cc:182] Had failures, leaving test files at /tmp/dist-test-taskeGzcyx/test-tmp/replace_tablet-itest.0.ReplaceTabletITest.ReplaceTabletsWhileWriting.1736935330657694-7644-0
[ FAILED ] ReplaceTabletITest.ReplaceTabletsWhileWriting (8046 ms)
[----------] 1 test from ReplaceTabletITest (8046 ms total)
[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (8046 ms total)
[ PASSED ] 0 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] ReplaceTabletITest.ReplaceTabletsWhileWriting
1 FAILED TEST